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ABSTRACT 


Within  the  Department  of  Defense's  (DoD's)  planning/ 
progreunming,  and  budgeting  system  (PPBS) ,  the  role  of  pro¬ 
gramming  is  to  transform  defense  needs  into  a  time-phased 
program  of  affordable  and  achievable  defense  activities. 

This  involves  decisions  on  priorities  among  the  potential 
activities,  since  not  all  can  be  afforded;  and  such  decisions 
require  professional  military  as  well  as  civilian  government 
judgments  about  the  relative  importance  of  various  defense 
needs.  Thus  the  programming  phase  of  PPBS  is  the  critical 
step  in  which  professional  judgment  is  matched  to  resources 
to  create  the  tangible  objectives  of  the  DoD.  Programming 
is  complicated  by  the  magnitude  of  the  program  (several 
thousand  elements  and  subelements)  and  the  diversity  of  the 
judgments  to  be  utilized. 

To  strengthen  its  programming  system  within  the  DoD 
PPBS,  the  Marine  Corps  (MC)  recently  has  introduced  and 
tested  new,  more  efficient  and  reliable  techniques  for  col¬ 
lecting  and  utilizing  the  professional  judgm.ents  of  its 
staff.  The  new  techniques  have  been  developed  using  the 
management  discipline  of  decision  analysis.  The  result  is 
a  practical,  conceptually  advanced  methodology  that  has 
succeeded  in  developing  consensus  among  the  MC  staff  where 
other  procedures  have  failed.  Based  on  its  successes  over 
the  past  four  years,  the  MC  has  brought  more  and  more  of  its 
program  under  the  management  of  its  central  programming 
system. 

The  MC  staff  continues  to  search  for  and  find  signifi¬ 
cant  new  ways  to  apply  decision  analysis  to  its  programming 
system.  However,  the  current  strategy  and  procedures  for 
decision-analytic  support  of  programming  have  reached  a 
level  of  sophistication  that  warrants  documentation  in  the 
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interests  of  the  continuity  of  prograiraning .  Moreover,  such 
documentation  may  suggest  ideas  to  the  other  military  services 
that  will  help  them  to  refine  their  own  programming  systems. 

This  report  describes  the  decision-analytic  methodology 
used  by  the  MC  in  its  progratrar.ing  system.  It  provides  a 
working  knowledge  of  the  methodology  and  of  MC  programming, 
not  merely  overviews.  It  is  suitable  as  a  reference,  and  it 
may  be  used  for  training.  Section  1.0  describes  the  purpose 
of  the  report.  Section  2.0  describes:  (1)  the  general 
system  of  military  programming  in  the  Department  of  Defense; 
and  (2)  the  MC ' s  approach,  including  the  MC ' s  goals  for  pro¬ 
gramming,  phases  of  programmiing ,  and  the  MC  purposes  that 
decision-analytic  methods  serve.  Section  3.0  describes  the 
prioritization  stage  of  MC  programming,  emphasizing  the  MC ' s 
decision-analytic  strategy  and  staff  guidelines  for  prioriti¬ 
zation.  Section  4.0  describes  the  decision-analytic  methods 
used  by  the  MC  for  prioritization,  including  staff  roles  and 
organization,  theory,  and  application  procedures.  Section  5.0 
provides  a  brief  summ>ary  and  conclusion. 
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DECISION-ANALYTIC  SUPPORT  OF  THE 
UNITED  STATES  MARINE  CORPS'  PROGRAM  DEVELOPMENT: 
A  GUIDE  TO  THE  METHODOLOGY 


1.0  INTRODUCTION 


1 , 1  Purpose 

This  report  describes  the  decision-analytic  methodology 
used  by  the  Marine  Corps  (MC)  to  develop  its  portion  of  the 
Department  of  the  Navy's  (DON’s)  program  objectives  memo¬ 
randum.  (POM) . 

This  report  has  two  companion  reports  that  document  a 
POM  data  base  managem.ent  system  (DBMS).^  This  system  sup¬ 
ports  the  MC  staff  as  it  collects,  prioritizes,  and  programs 
the  items  of  the  POM  using  the  methodology  described  here. 
The  three  documents  form,  a  package  that  explains  the  nature 
of  the  MC  staff's  methodology  as  of  November  1980.  This 
package  can  be  used  to  solicit  decision-analytic  contractor 
support  for  the  MC  staff's  POM  development,  or  to  develop 
such  support  within  the  MC  staff.  In  either  case,  it  is  as¬ 
sumed  that  the  users  of  this  report  have  a  broad  background 
in  the  decision  sciences.  The  report  indoctrinates  these 
users  in  the  aspects  of  decision  analysis  that  have  been 
applied  to  the  MC ' s  development  of  the  POM. 


Waslov,  K.  A.;  Kuskey,  K.  P.,  Program.  Objectives  Memorandum- 
Data  Base  Management  System  Users  Manual  UM-81-1-158  (McLean 
Virginia :  Decisions  and  Designs,  Inc . ,  May  1981). 

Esoda,  R.  M. ,  Program  Objectives  Memorandum-Data  Base 
Management  System  Program  Maintainance  Manual,  UM-81-2-158 
(McLean,  Virginia:  Decisions  and  Designs,  Inc.,  May  1981). 


1 . 2  Background 

The  MC  POM-development  process  consists  of  an  annual 
reassessment  and  adjustment  of  the  Commandant  of  the  Marine 
Corps's  (CMC's)  program  for  the  MC.  Over  the  years,  the  MC 
staff's  strategy  and  methods  for  preparing  the  MC ' s  POM  sub¬ 
mission  to  the  Don  have  evolved  substantially.  POM  prepara¬ 
tion  has  become  the  vehicle  for  a  systematic,  comprehensive 
management  review  of  MC  missions,  manpower,  operations,  and 
acquisitions . 

A  significant  factor  in  the  evolution  of  MC  programmiing 
has  been  the  introduction  of  the  management  discipline  of 
decision  analvsis.  The  MC  staff  first  explored  the  usefulness 
of  decision  analysis  for  POM  development  in  a  project  spon¬ 
sored  jointly  with  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  in  1977.  In  each  subsequent  year  the  MC 
staff  has  obtained  contractor  support  to  continue  the  develop¬ 
ment  and  implementation  of  decision-analytic  programming 
methods.  The  bibliography  in  Section  2.4  lists  a  series  of 
reports  through  which  the  evolution  of  MC  POM  methodology 
can  be  traced.  This  report  describes  the  current  status  of 
the  methodology. 

The  MC  staff's  application  of  decision  analysis  to  the 
POM  is  distinctive  for  its  twin  emphases  on  the  judgmental 
and  group  aspects  of  MC  decision  making.  At  root,  it  is  a 
scheme  for  finding  "wisdom  in  a  multitude  of  counselors." 

By  dividing  the  MC  staff  into  working  groups  with  special 
expertise,  eliciting  these  groups'  best  collective  judgments 
about  progr2un  priorities,  and  then  merging  their  judgments, 
overall  MC  priorities  emerge.  Decision-analytic  methods  are 
used  to  effect  this  divide-conquer-reunite  process.  They 
involve  (1)  the  quantitative  representation  of  the  officers' 
judgments  of  priority,  (2)  a  "sampling"  procedure  applied  to 
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the  groups  to  merge  their  judgments,  and  (3)  a  cost-effectiveness 
analysis  based  on  the  quantified  judgments  of  priority. 


While  the  introduction  of  decision-analytic  methods  has 
had  a  pervasive  effect  on  the  process  of  >’-C  programming,  the 
methods  themselves  are  developed  and  applied  by  only  a  hand¬ 
ful  of  individuals  within  the  Program  Coordination  Branch 
(RPP)  of  the  Deputy  Chief  of  Staff  for  Requirements  and  Pro¬ 
grams  (DC/S  R&P) .  RPP  has  used  decision  analysis  to  advance 
the  MC  staff  consistently  towards  three  major  goals: 

(a)  A  systematic,  practical  MC-staff  process  of  pro¬ 
gramming,  extending  through  the  full  POM  cycle 
from  initial  program  guidance  to  amended  program- 
decision  memorandum,  that  assists  the  MC  staff  to 
satisfy  MC  m.ilitary  needs  with  programs  in  a  com.- 
pellingly  logical  way. 

(b)  Sound  procedures  for  eliciting,  representing,  com,- 
bining,  and  using  the  collective  professional 
military  judgments  of  the  MC  staff  to  determine 
program,  priorities. 

(c)  Timeliness,  accuracy,  and  flexibility  of  program, - 
ming--plus  an  audit  trail  of  all  judgments  of 

pr iority--achieved  through  a  specialized  decision 
aid  and  data  base  management  system,  that  supports 
the  decision-analytic  POM  methodology. 

The  MC  methodology  is  a  unique  application  of  decision 
analysis  that  advances  the  state  of  the  art  for  practical 
resource  allocation  and  programming  in  large  institutions. 
Besides  the  MC  POM,  it  has  been  applied  to  (1)  the  U.S.  Army 
POM,  (2)  the  Army's  mobility  equipment  research  and  develop¬ 
ment  program,  (3)  the  Tactical  Air  Command's  program,  for 
night  and  weather  acquisitions,  (4)  the  National  Aeronautic 
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and  Space  Administration's  research  program,  and  (5)  commer¬ 
cial  strategies  for  product  development.  References  are 
found  in  Section  2.4. 


1.3  Plan 


Sections  2.0  and  3.0  describe  the  practical  goals, 
scientific  background,  and  overall  logic  of  the  MC ' s  decision- 
analytic  methodology  for  POM  development.  Section  4.0  describes 
the  technical  procedures  for  carrying  out  the  methodology. 

Both  sections  are  written  for  the  reader  who  needs  a  working 
knowledge  of  the  methodology,  not  just  an  overview. 


2.0  BACKGROUND  AND  CONCEPTUAL  FRAMEWORK 


2.1  Introduction 


This  section  describes  decision-analytic  support  in  the 
overall  context  of  MC  planning,  prograrrjning,  and  budgeting; 
formulates  the  MC  prograimning  problem  from  a  methodological, 
resource-allocation  perspective;  and  provides  a  bibliography 
of  applications  and  theory. 

2 . 2  The  General  Context  of  Decision-Analytic  Support 

Various  broad  aspects  of  the  DoD,  DoN,  and  MC  planning, 
programming,  and  budgeting  systems  (PPBSs)  are  important 
background  for  designing  and  applying  decision-analytic 
methods  to  MC  POM  development.  These  are  discussed  in  terms 
of  the  federal  budgetary  system,  DoD  PPBS,  Navy/MC  PPBS ,  and 
MC  PPBS. 

2.2.1  Federal  budgetary  system  -  All  federal  expendi¬ 
tures  are  determined  during  the  federal  government's  annual 
budget  cycle.  Under  law,  the  President's  role  is  to  develop 
and  to  propose  to  Congress  a  unified  plan  (budget)  of  expen¬ 
ditures  every  year,  while  the  role  of  the  Congress  is  to  re¬ 
view  this  plan,  develop  alternatives,  authorize  programs, 
appropriate  funds,  and  legislate  taxes  to  provide  funds. 

The  President's  staff  for  the  development  of  the  budget  is 
the  Office  of  Management  and  Budget  (0MB) .  0MB  forms  the 
President's  budget  from  proposals  submitted  by  all  the 
federal  agencies,  including  the  defense  agencies.  The 
President's  proposed  budget  for  defense  is  a  product  of  the 
Office  of  the  Secretary  of  Defense  (OSD)  and  0MB  working 
jointly. 
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2.2.2  DoD  PPBS  -  To  develop  a  defense  budget  for  the 
President,  OSD  manages  the  development  of  a  joint-service 
budget  through  its  PPBS.  The  PPBS  is  internal  to  DoD  and 
does  not  interface  with  0MB  or  Congressional  systems. 

It  takes  approximately  two  years  to  develop  the 
DoD's  budget  with  the  PPBS  system,  and  then  another  year  for 
Congressional  approval.  The  first  year  consists  chiefly  of 
long-range  planning  by  the  Joint  Chiefs  of  Staff  (JCS) .  The 
JCS  addresses  threats  posed  to  the  United  States  through  the 
next  twenty  years.  It  formulates  the  directions  and  empha¬ 
ses  that  the  services  should  follow  to  meet  these  threats. 

Early  in  the  year  ( January-February) ,  the  JCS  publishes  the 
Joint  Strategic  Planning  Document  and  Supporting  Analyses, 

Book  I  (JSPDSA  1),  which  consists  of  threat  appraisal, 
military  objectives  and  strategy,  and  force  planning  guidance 
to  the  services.  After  feedback  is  received  from  the  services 
and  analyzed,  the  JSPDSA  II  is  published  near  the  end  of  the 
year;  it  assesses  the  "reasonable  assurance"  of  successfully 
executing  the  national  military  strategy.  An  executive 
sximmary  of  the  JSPDSA  I  and  II,  called  the  JSPD,  is  forwarded 
to  the  SECDEF  in  November. 

Based  on  the  JSPD  and  other  inputs  from  the 
President,  the  National  Security  Council,  and  the  OSD  staff, 
the  SECDEF  publishes  its  initial  version  of  the  Draft  Con¬ 
solidated  Guidance  (CG)  to  the  services  in  the  January/February 
time  frame.  The  initial  Draft  CG  marks  DoD's  transfer  from 
conceptual  planning  to  programmatic  planning.  After  review 
by  the  services  and  the  JCS,  the  final  Draft  CG  is  published 
in  March.  (The  CG  is  always  in  a  "draft"  status.) 

The  CG  tells  the  services  the  types  of  programs 
and  force  structure  they  should  develop  over  the  next  several 
years  to  support  the  national  military  strategy.  The 
guidance  is  both  fiscal  and  programmatic.  It  sets  limits  on 
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the  dollar  sums  of  the  services'  budgets,  and  it  directs 
that  certain  programs  or  forces  should  be  provided.  This 
guidance  is  in  a  multi-level,  multi-year  format.  It  speci¬ 
fies  three  different  program  sizes  (minimum,  basic,  and 
enhanced)  for  each  service  in  terms  of  total  dollar  expendi¬ 
tures  in  each  of  five  years.  It  directs  that  certain  programs 
should  be  programiried  in  the  minimum,  basic,  or  enhanced 
levels . 


The  planning  effort  that  concludes  with  the 
Draft  CG  is  followed  by  a  year  of  programming  and  budgeting. 
First,  the  services  use  the  Draft  CG  of  March  to  revise 
their  overall  programs.  In  May,  they  each  submit  a  POM  to 
the  SECDEF  that  shows  what  they  propose  to  program  with  the 
three  different  funding  levels  of  the  CG.  The  SECDEF  then 
develops  an  overall  defense  program,  from  the  POMs  (and  from 
the  program,  submissions  of  the  defense  agencies,  e.g.,  the 
Defense  Communications  Agency) .  The  three-level  POMs  pro¬ 
vide  the  SECDEF  with  the  flexibility  (1)  to  adjust  the 
defense  budget  from  approximately  5%  below  current  expendi¬ 
tures  to  5%  above,  to  reflect  the  emphasis  that  0MB  and  the 
President  choose  to  pursue;  and  (2)  to  redirect  the  emphasis 
of  defense  among  the  services  to  carry  out  the  national 
military  strategy.  The  services’  development  of  the  POMs  is 
discussed  at  more  length  in  Section  2.3.1. 

At  the  OSD  level,  the  development  of  the  defense 
programi  from  the  POMs  proceeds  as  follows:  first,  the  JCS 
staff  studies  the  POMs  to  see  how  well  they  satisfy  the  JSPD 
and  CG;  their  review  is  forwarded  to  OSD.  Second,  the  OSD 
staff  reviews  the  POMs  to  judge  their  congruence  to  the  CG. 
The  result  of  both  reviews  is  a  set  of  issue  papers  (IPs) 
prepared  by  OSD  that  challenges  various  features  of  the 
POMs.  The  services  and  the  JCS  respond  to  these  IPs.  Then 
the  SECDEF  publishes  the  Program  Decision  Memorandimn  (PDM) , 
the  SECDEF 's  proposed  defense  program.  After  evaluating 
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specific  objections  to  the  PDM  from  the  services,  the  SECDEF 
publishes  the  Amended  PDM  (APDM)  in  September,  and  this 
marks  the  end  of  the  programming  phase. 


The  services  next  present  budgets  to  OSD  for 
implementing  their  approved  programs  in  the  APDM.  These  are 
reviewed  and  revised  by  OSD  and  0MB  and  become,  collectively, 
the  SECDEF 's  budget  proposal  to  0MB  and  the  President.  The 
President  presents  his  budget  to  the  Congress  in  January, 
two  years  after  the  publishing  of  the  JSPDSA  I,  and  one  year 
after  the  publishing  of  the  Draft  CG.  The  Congress  has  nine 
months  after  the  President  has  presented  his  budget  to  review 
it  and  enact  the  federal  budget  for  the  fiscal  year  beginning 
October  1.  In  recent  years,  the  Congress  has  taken  twelve 
months  to  accomplish  this,  rather  than  nine.  Thus,  the 
period  from  JSPDSA  I  to  federal  budget  is  three  years. 

This  discussion  of  DoD  PPBS  has  focused  on  cer¬ 
tain  milestones  that  occur  during  preparation  of  the  DoD 
budget:  JSPDSA  I,  Draft  CG,  the  POMs,  and  the  APDM.  The 
PPBS  is  roughly  analogous  to  a  relay  race:  on  a  race  track, 
the  milestones  mark  the  passing  of  the  baton  from  one  runner 
to  another.  However,  the  PPBS  relay  has  some  unusual 
features.  First,  the  runners  not  only  pass  the  baton  to  the 
next  runner,  they  also  receive  another  baton  and  keep  running. 
Though  they  get  some  rest,  the  planners  never  stop  planning, 
the  programmers  never  stop  programming,  and  the  budgeteers 
never  stop  budgeting.  They  work  in  one-year  cycles,  and 
once  a  year  they  pass  off  their  year's  work  to  other  workers 
and  pick  up  their  next  year's  work  from  yet  other  workers. 
Second,  unlike  relay  runners,  the  PPBS  runners  start  running 
to  catch  the  next  baton  well  before  it  is  ready  to  be  passed. 
The  programmers  have  done  four  months'  work  before  the  Draft 
CG  is  issued  so  that  they  will  be  prepared  to  follow  it. 

The  budgeteers  use  the  PDM  to  start  work  so  that  they  can 
respond  quickly  to  the  APDM.  The  system  simply  would  not 


function  if  the  programmers  and  budgeteers  did  not  have 
themselves  fully  in  motion  as  the  Draft  CG  and  APDM  were 
passed  to  them. 

A  cornerstone  of  DoD  PPBS  that  provides  coordi¬ 
nation,  continuity,  and  visibility  to  the  defense  program,  is 
the  SECDEF's  Five-year  Defense  Program  (FYDP) .  It  exists  as 
a  data  base  containing  (1)  historical  data  on  the  defense 
program  as  executed  since  1962;  (2)  the  current  program  as 
authorized  and  appropriated  by  Congress;  and  (3)  the  multi¬ 
year  program  for  the  future  that  evolves  during  the  PPBS. 

The  FYDP '  s  multi-year  programi  is  updated  three  times  every 
year  to  reflect  (1)  the  POMs  submitted  by  the  services 
(May) ;  (2)  the  budgets  submitted  to  0MB  by  the  services  and 

OSD  (October);  and  (3)  the  President's  budget  (January). 

2.2.3  Navy/MC  PPBS  -  Because  the  Marine  Corps  is  a 
part  of  the  Department  of  the  Navy,  its  program  is  developed 
as  part  of  the  DoN  POM  rather  than  as  an  independent  MC  POM. 
However,  a  division  of  the  DoN's  PPBS  between  the  MC  and 
Navy  staffs  has  been  established  by  the  Secretary  of  the 
Navy  (SECNAV) .  In  this  division,  the  MC ' s  PPBS  for  MC  per¬ 
sonnel  and  ground-warfare  capabilities  is  virtually  autono¬ 
mous  fromi  the  Navy  staff's  PPBS,  while  the  MC  PPBS  effort 
for  air  warfare  and  other  program  areas  occurs  jointly  with 
the  Navy  staff.  This  division  is  subject  to  change;  for 
instance,  military  construction  to  support  the  MC  has  re¬ 
cently  come  within  the  autonomous  MC  PPBS .  For  the  autono¬ 
mous  efforts,  MC  staff  operates  under  separate  fiscal  and 
programmatic  guidance  from  the  SECNAV.  Under  this  guidance, 
it  develops  the  so  called  "green-dollar  POM"  which  the 
SECNAV  then  incorporates  into  the  DoN  POM. 

Fiscal  guidance  for  the  MC-developed  (green 
dollar)  portion  of  the  DoN  POM  is  determined  by  the  SECNAV. 
As  a  rule,  this  fiscal  guidance  for  the  MC  is  obtained  by 
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multiplying  the  CG's  guidance  to  the  DoN  by  a  fraction.  The 
fraction  is  a  ratio  of  current  MC  expenditures  to  total  DoN 
expenditures.  The  fraction  is  a  very  important  determinant 
of  the  MC's  program,  and  hence  it  is  defined  by  the  SECNAV 
in  consultation  with  the  Commandant  of  the  Marine  Corps 
(CMC)  and  the  Chief  of  Naval  Operations  (CNO) .  It  deter¬ 
mines  the  so  called  "blue-green  split"  that  divides  the  CG's 
multi-level,  multi-year  dollar  totals  for  the  DON  between 
the  CNO  and  CMC's  programmers.  Given  this  split,  the  HQMC 
builds  its  independent  portion  of  the  DoN  POM. 

However,  as  already  mentioned,  a  substantial 
portion  of  the  DoN  POM  that  is  related  to  the  MC  is  not  pro- 
grautuned  by  the  CMC  because  it  is  tied  closely  to  the  CNO's 
program.  This  is  true  of  MC  aviation  procurement,  for  in¬ 
stance,  and  in  this  case,  the  CMC  proposes  an  aviation  pro¬ 
gram  to  the  Navy  staff.  The  Navy  staff  develops  the  related 
CNO's  POM  submission.  The  same  procedure  is  followed  for 
the  Marines'  research,  development,  test,  and  evaluation 
(RDT&E)  activities,  which  are  part  of  the  CNO's  POM  submis¬ 
sion. 

2.2.4  MC  PPBS  -  As  discussed  above,  the  SECNAV  assigns 
part  of  the  DoN  PPBS  to  the  CMC.  Generally,  the  CMC  pro¬ 
vides  all  planning,  prograiriming,  and  budgeting  related  to  MC 
manpower,  base  operations,  fleet  support,  military  construc¬ 
tion,  and  ground-warfare  readiness  and  modernization.  The 
CMC  delegates  planning  to  the  DC/S  for  Plans,  Policies,  and 
Operation  (PPO) ;  programming  to  the  DC/S  for  Requirements 
and  Programs  (R&P) ;  and  budgeting  to  the  Fiscal  Director  of 
the  Marine  Corps  (FD) . 

HQMC's  PPBS  is  identical  in  its  broad  features 
to  the  PPBSs  of  the  Army,  Air  Force,  and  Navy.  In  the 
planning  phase,  DC/S  PPO's  staff  makes  use  of  JCS  and  OSD 


intelligence  assessments  to  revise  long-  and  short-range  MC 
plans  and  policies;  and  it  interacts  with  OSD  in  shaping  the 
CG.  In  the  programming  phase,  DC/S  R&P's  staff  reviews  and 
revises  the  MC's  force  structure;  it  reviews  current  opera¬ 
tions  and  acquisitions,  requests  program  initiatives  from 
the  MC  staff,  prioritizes  these  with  the  current-capability 
items,  and  packages  the  result  into  the  green-dollar  POM;  it 
responds  to  issue  papers  in  shaping  the  APDM.  In  the  bud¬ 
geting  phase,  the  FD  staff  divides  the  POM  into  budget 
bands,  puts  it  in  budget  format,  and  takes  it  through  re¬ 
views  by  the  Comptroller  of  the  Navy,  OSD,  and  0MB. 

The  programrinc  phase  of  the  MC’s  green-dollar 
PPBS  and,  more  narrowly,  the  MC's  strategy  and  methodology 
for  prograrriming ,  are  described  in  detail  in  later  sections. 
Several  broad  features  of  the  programir.ing  effort  are  as 
follows : 

(1)  Staff  committees  -  During  the  pre^  .ration  of  the 
green-dollar  POM,  two  ad  hoc  comm.ittees  assist 
DC/S  R&P.  These  are  the  POM  Working  Group  (PWG) , 
composed  of  action  officers  representing  the  MC 
sponsors  (i.e.,  the  major  staff  offices);  and  the 
POM  Coordinating  Group  (PCG) ,  composed  of  General 
Officers  representing  the  sponsors.  The  green- 
dollar  POM  is  reviewed  and  revised  by  the  PWG,  the 
PCG,  and  the  Chief  of  Staff's  Committee  before 
reaching  the  Commandant. 

(2)  Mission-area  format  -  For  several  years,  the  Con¬ 
gress,  0MB,  and  DoD  have  sought  to  emphasize  mili¬ 
tary  "missions"  in  establishing  the  defense  budget. 
Both  the  Congressional  Budget  Act  of  1974  and  0MB 
Circular  A-109  require  the  DoD  to  display  budget 
resources  in  terms  of  mission  areas.  Consistent 
with  this  emphasis,  the  MC  staff  uses  mission  areas 
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as  the  structure  for  organizing  all  aspects  of  its 
green-dollar  programming  effort.  This  mission- 
area  emphasis  in  programming  proceeds  through  four 
stages : 

(a)  the  definition  of  mission  areas; 

(b)  the  determination  of  MC  capabilities  and  de¬ 
ficiencies  within  the  areas  relative  to  the 
current  and  future  war  threats; 

(c)  the  solicitation  and  prioritization  of  pro¬ 
gram  initiatives  to  decrease  the  deficiencies 
and 

(d)  the  packaging  of  initiatives  into  sets  that 
solve  the  mission  deficiencies  in  a  balanced 
fashion . 

The  m.issions  for  POM  8  3  are  shown  in 
Figure  2-1.  The  horizontal  lines  in  the  figure  signify 
clusters  of  mission  areas  that  fall  naturally  to  one  or 
another  of  the  MC  sponsors.  The  sponsors  provide  the  mili¬ 
tary  expertise  for  prioritizing  program  initiatives  within 
and  across  the  mission  areas  in  the  clusters;  the  DC/S  R&P 
provides  the  expertise  for  balancing  the  priorities  of 
initiatives  across  the  clusters. 

(3)  Decision-analytic  orientation  -  The  CMC  uses  the 

PPBS  as  a  primary  management  system.  The  program¬ 
ming  phase  is  the  CMC's  forum  for: 


(a)  determining  goals  and  shortfalls  related  to 
USMC  missions; 
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(b)  discovering  options  for  new  action; 

(c)  analyzing  the  options  to  discover  their 
strengths  and  weaknesses  relative  to  one 
another  and  to  current  actions;  and 

(d)  selecting  a  program  of  new  and  current  ac¬ 
tions. 

As  such,  the  development  of  the  POK  is  the  CMC ' s  primary 
management  process  for  defining,  analyzing,  and  selecting 
the  multitude  of  program  decisions  needed  to  develop  the 
USMC's  capabilities  in  a  balanced,  integrated  manner.  That 
is  its  function  and  its  orientation. 

2 . 3  Conceptual  Framework  for  the  Application  of  Decision 

Analysis  to  the  MC ' s  POM  Development 

The  purpose  of  this  section  is  to  define  the  Marines' 
programming  problem  from  a  methodologist's  perspective, 
i.e.,  to  cast  it  in  terms  that  suggest  the  decision-analytic 
methods  that  can  be  applied  to  it  most  effectively.  The 
section  begins  with  some  basic  features  of  the  MC  program¬ 
ming  problem.  The  MC ' s  general  approach  to  programming  is 
characterized  as  a  method  that  aims  to  simultaneously 
(1)  promote  a  larger  budget  to  fulfill  unmet  military  re¬ 
quirements,  and  (2)  rationally  allocate  its  budget  to  pro¬ 
duce  as  strong  a  military  capability  as  possible  within 
resource  constraints.  The  steps  in  this  approach  are  out¬ 
lined.  Among  them,  the  step  of  developing  an  MC  staff  con¬ 
sensus  on  the  relative  importance  of  all  projects  competing 
for  programming  is  identified  as  the  key  step  for  decision- 
analytic  support. 

2.3.1  Description  of  the  programming  problem  -  The 
general  programming  problem,  of  which  the  Marines'  is 
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typical,  is  a  dual  problem.  It  is  to  plan  how  to  (1)  obtain 
resources  and  (2)  optimally  allocate  these  resources  to  ac¬ 
tivities  and  acquisitions  if  they  are  obtained.  These  twin 
features  of  the  problem  are  inextricable  since  the  annual 
progr2un  submission  is  simultaneously  a  proposal  to  obtain 
resources  and  a  commitment  on  how  to  allocate  whatever  re¬ 
sources  are  obtained.  Inevitably,  the  proposal  to  obtain 
more  resources  is  judged  in  light  of  the  overall  plan  for 
spending . 


The  problem  has  many  features  common  to  standard 
resource  allocations  in  which  a  knovrn  but  limited  resource 
(e.g.  man  hours,  investment  dollars,  operating  dollars,  or 
facility  space)  must  be  optimally  apportioned  to  a  set  of 
projects  or  programs  competing  for  it,  realizing  that 

o  each  project  can  be  given  various  levels  of  re¬ 
source  corresponding  to  different  levels  of  opera¬ 
tion  or  investment; 

o  because  of  limited  resources,  not  every  project 

can  be  given  the  total  resources  it  could  reason¬ 
ably  use;  and 

c  the  projects  have  varying  degrees  of  importance 

based  on  fulfilling  different  broad  institutional 
goals. 

For  such  problems,  the  general  solution  is  to  develop  rela¬ 
tive  measures  of  cost  and  benefit  for  each  level  of  each 
project.  From  these  one  determines  the  most  cost-effective 
allocation  of  resources  to  the  projects,  i.e.,  an  allocation 
that  gives  the  most  benefit  for  the  amount  of  resources 
expended.  Such  analyses  take  many  forms  and  are  applied  to 
many  kinds  of  problems,  ranging  from  the  design  of  new 
products  to  the  design  of  multi-party  negotiations. 
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The  identification  of  projects  and  the  deter¬ 
mination  of  their  costs  and  benefits  are  key  aspects  of  POM 
development.  However,  because  the  MC  staff  is  not  merely 
allocating  resources  it  already  has,  but  requesting  re¬ 
sources,  straightforward  cost-benefit  analysis  must  be  tem¬ 
pered  with  other  considerations  in  arriving  at  the  program 
to  propose.  First  and  most  importantly,  external  competitor 
for  resources  can  win  an  MC  project's  proposed  resources  if 
one  of  their  projects  gives  more  value  for  the  resources  ex¬ 
pended.  Therefore,  the  MC's  projects  must  look  as  valuable 
as  possible  to  improve  the  likelihood  that  they  will  be  ac¬ 
cepted  into  the  FYDP  and  the  DoD's  budget.  Unfortunately,  a 
cost-effective  POM  submission  can  appear  weaker  than  it 
really  is:  compared  to  less  cost-effective  submissions,  it 
will  usually  have  more  lower-cost  projects  of  lesser  impor¬ 
tance,  though  in  aggregate  they  are  more  valuable.  Specula¬ 
tively,  a  large  number  of  small  projects  is  more  difficult 
for  the  DoD  to  evaluate  and  understand  than  a  sm.all  number 
of  larger,  more  important  projects. 

A  second  feature  complicating  a  straightforward 
resource  allocation  approach  to  procrramming  is  the  -ulti- 
level  nature  of  the  program.  The  MC's  POM  submission  m.ust 
show  the  CMC '  s  program,  for  three  increasing  levels  of  over¬ 
all  funding  specified  by  the  SECNAV.  These  are  the  minimum., 
basic,  and  enhanced  levels,  respectively.  On  its  own  initia 
tive,  the  MC  may  also  propose  to  spend  an  "over  guidance" 
amount  beyond  the  enhanced  level.  This  defines  an  "over¬ 
guidance  level"  for  the  submission.  Projects  that  are  not 
programjned  in  one  of  these  four  levels  are  said  to  be  in  the 
"unfunded  level." 

Several  features  of  the  multi-level  format 
impact  methodology: 
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(1)  The  minimum-level  program  is  protected  from  resource 
competition.  As  a  rule,  the  MC  and  the  other  ser¬ 
vices  are  guaranteed  their  minimum-level  programs. 
This  means  that  the  minimum  level  can  be  used  to 
protect  programs.  It  creates  an  incentive  for  the 
services  to  put  some  weaker  programs  under  protec¬ 
tion  and  risk  some  stronger  programs  in  the  basic 

or  enhanced  levels  as  tools  for  obtaining  more 
funding . 

(2)  The  basic-level  program  must  be  presented  as  a  set 
of  projects  that  merely  builds  upon  the  minimum- 
level  program;  and  the  enhanced-level  program,  is  a 
set  of  projects  that  build  upon  the  basic-level 
program,.  Contrary  to  this  formiat,  a  resource  al¬ 
location  based  on  cost-benefit  analysis  could  ex¬ 
clude  projects  at  the  basic  level  which  it  included 
at  the  minimum,  level . 

(3)  The  basic  and  enhanced  programs  m.ust  be  "packaged" 
into  decision  units  (consolidated  decision  package 
sets)  for  the  SECNAV.  These  packages  (normally 
not  more  than  thirty  for  the  MC) ,  rather  than  in¬ 
dividual  projects,  will  compete  for  resources.  To 
make  up  packages  that  are  understandable  as  uni¬ 
fied  wholes,  sim.ilar  projects  will  tend  to  be 
packaged  together  even  though  they  vary  widely  in 
importance  and  cost  effectiveness  and  would  not  be 
put  together  in  a  straightforward  resource  alloca¬ 
tion  . 

2.3.2  Approach  to  the  programming  problem  -  Given  the 
foregoing  features  of  MC  programming,  the  MC  staff's  approach 
to  the  problem  is  a  judicious  mixture  of  broad  strategy, 
systematic  procedures,  scientific  techniques,  and  ad  hoc 
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problem  solving.  This  section  outlines  these  elements  of 
their  approach. 


2. 3. 2.1  Goals  for  programming  -  As  to  broad 
strategy,  the  CMC's  first  programming  priority  is  to  assure 
that  the  minimum-level  program  defines  a  functional,  bal¬ 
anced  Marine  Corps,  even  though  this  means  (by  definition) 
operating  with  five  percent  less  than  current  funding.  This 
is  the  CMC's  first  priority  because  experience  has  shown 
that  the  Marines  cannot  count  on  obtaining  the  basic-level 
program,  though  they  almost  alv;ays  obtain  the  minimum-level 
program..  This  first  priority  leads  to  great  care  in  pre¬ 
paring  the  minimum-level  program.  Its  preparation  is  based 
upon  a  systematic  review  and  adjustment  of  Marine  Corps  mis¬ 
sions,  manpower  (military  and  civilian),  and  current  acquisi¬ 
tions.  New  acquisitions  m.ay  be  incorporated  into  the  minimum 
level  to  the  extent  they  are  m.ore  important  than  current  ac¬ 
quisitions  or  operations,  but  they  seldom  are  m.ore  important. 

Broadly  spea)<:ing,  the  cr'.C's  second  pro- 
gramm.ing  priority  is  to  construct  a  basic-level  program  that 
maintains  or  improves  near-term  readiness  through  the  re¬ 
placement  or  overhaul  of  obsolete  or  unserviceable  equip- 
m.ent,  the  maintenance  of  equipm.ent,  and  the  maintenance  of 
facilities.  More  important  issues  of  readiness,  such  as 
recruitment  and  training,  are  included  in  the  minimum.-level 
program.  Some  new  capabilities  may  be  programmed  at  the 
basic  level,  but  not  many. 

The  CMC's  third  programming  priority  is 
to  construct  an  enhanced-level  program  that  will  improve  the 
MC's  readiness  in  the  future  through  the  application  of  cur¬ 
rent  engineering  technology  to  established  MC  requirem.ents . 
These  applications  would  add  or  extend  current  MC  capabili¬ 
ties  significantly. 
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2. 3. 2. 2  Phases  of  prograramin 


The  POM  submis¬ 


sion  results  from  a  set  of  parallel  program  developments 
initiated  and  coordinated  by  DC/S  R&P.  The  schedule  in 
Figure  2-2  (a  and  b)  shows  the  parallel  milestones  of  the 
POM  83  cycle  that  control  the  development  of  a  military 
manpower  program,  a  civilian  manpower  program,  a  procurement 
program,  an  operations  and  maintenance  {O&M)  program,  the 
POM  submission,  and  a  mission  area  analysis.  The  parallel 
program  developments  can  be  viewed  from  several  perspec¬ 
tives  . 

First,  they  divide  the  developm^ent  of 
the  POM  submission  into  two  ordered  phases:  (1)  the  m.ilitary 
and  civilian  manpower  programs  and  the  "constrained  minimum- 
level"  O&xM  program  are  finalized  first:  then  (2)  the  pro- 
curemient  and  "unconstrained"  O&M  program,  are  finalized  after¬ 
wards.  The  first  phase  establishes  a  "constrained  minimum- 
level"  core  program  that  requires  less  resource  than  the 
minimum,  level;  the  second  phase  builds  from,  the  core  up 
through  minimum,  basic,  enhanced,  and  over-guidance  levels. 
Given  the  strategy  described  above,  the  first  phase  is 
considered  more  critical  than  the  second. 

Second,  in  both  these  phases  the  proce¬ 
dures  order  staff  action  through  the  same  sequence  of 
stages:  (1)  a  request  for  program,  initiatives;  (2)  a  pri¬ 

oritization  of  initiatives;  and  (3)  the  construction  of  a 
program,  from  the  prioritized  list.  These  stages  set  the 
inner  rhythm  of  POM  development.  They  vary  markedly  in  MC 
participation:  (1)  the  request  for  initiatives  is  MC-wide; 

(2)  the  prioritization  is  only  Headquarters  MC-wide;  and 

(3)  the  construction  of  the  program  is  effectively  limited 
even  further  to  the  POM  Working  Group  and  the  POM  Coordina¬ 
tion  Group.  They  vary  markedly  in  effect:  (1)  the  request 
produces  a  large  volume  of  initiatives;  (2)  the  prioritiza¬ 
tion  screens,  validates,  and  ranks  the  initiatives  in  terms 
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POM  83  SCHEDULE 
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of  military  worth;  and  (3)  the  construction  phase  pares,  re¬ 
orders,  and  repackages  the  initiatives  into  a  marketable, 
cost-effective  program  that  meets  DoD/DoN  fiscal  and  program¬ 
matic  guidance.  They  vary  markedly  in  character:  (1)  the 
request  is  a  relatively  open,  non judgmental  solicitation  of 
programs;  (2)  the  prioritization  is  an  intensely  evaluative, 
judgmental  process  that  examines  the  merits  of  programs  in¬ 
dividually;  and  (3)  the  construction  stage  is  marked  by  as¬ 
pects  of  pragmatic  programming  where  programs  become  mere 
elements  of  packages  and  must  be  trimmed  and  combined  in 
ways  that  make  the  packages  most  attractive. 

2. 3. 2. 3  Scientific  techniques  -  The  prioritiza¬ 
tion  stage  of  POM  development  involves  several  procedures 
that  draw  directly  on  psychological  and  economic  science. 
These  procedures  introduce  an  objective  logic  of  analysis 
and  evaluation  to  the  prioritization  to  enhance  its  quality, 
its  efficiency,  and  its  acceptability  to  the  MC  staff.  It 
is  a  logic  that  prescribes  how — in  general  terms  and  in  spe¬ 
cific  cases — to  assess  professional  military  judgment  about 
program  worth,  to  organize  and  integrate  the  collection  of 
these  judgments  efficiently,  and  to  use  the  results  to  set 
program  priorities.  Section  3.0  describes  this  logic,  while 
Section  4.0  describes  the  procedures  used  to  carry  it  out. 

2. 3. 2.4  1  hoc  problem  solving  -  Some  parts  of 
the  prioritization  stage  are  not  amenable  to  general  pre¬ 
determined  procedures  but  require  ad  hoc  problem  solving. 

The  reconciliation  of  professional  military  judgments,  for 
instance,  always  requires  an  approach  specific  to  the  prob¬ 
lem  and  personalities  at  hand.  Similarly,  the  packaging  of 
progreuns  during  the  construction  stage,  though  based  largely 
on  the  prioritization  stage,  must  be  adapted  to  the  specific 
programming  "environment"  the  Marines  find  themselves  in 
each  year,  i.e.,  to  the  particular  programs  and  priorities 
that  the  OSD,  OJCS,  and  the  other  services  are  advocating. 
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2.4.1  Government  manuals,  memoranda,  circulars  - 


Department  of  the  Navy  Programming  Manual, 

OPNAV  90- ID.  (Basic  Navy  PPBS  reference;  it 

is  revised  three  times  each  year;  source: 

OP  090. 

Headquarters  Order  P3121.2,  Marine  Corps  Manual 
for  Planning  and  Programming 

PPBS  Handbook,  U.S.  Army  Program  Analysis  and 
Evaluation  Directorate 

Joint  Chiefs  of  Staff,  Memorandum  of  Policy  No.  84, 
(Planning  phase  reference.) 

0MB  Circular  A-11,  Preparation  and  Submission 
of  Budget  Estimates 

DoD  Manual  7110-1-M,  Budget  Guidance  Manual 

NAVCOMPT  Manual,  Vol.  7,  Budgeting 

POM  Serials ,  Headquarters,  U.S.  Marine  Corps 

(Published  annually  to  coordinate  MC  program¬ 
ming  . ) 
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2.4.2  Development  and  application  of  the  MC  program- 
methodology  - 


Bresnick,  T.  A.,  and  Kuskey,  K.  P.  U.S.  Marine 
Corps  Aviation  POM-82  Program  Benefits  for 
Competitive  Programs^  Interim  Report  PR  80-6-150. 
McLean,  Virginia:  Decisions  and  Designs, 

Inc. ,  May  1980. 

Bresnick,  T.  A.,  and  Kuskey,  K.  P.  U.S.  Marine 
Corps  POM-82  Program  Benefits  for  Competitive 
Programs^  Interim  Report  PR  80-12-150. 

McLean,  Virginia;  Decisions  and  Designs, 

Inc. ,  June  1980. 

Buede,  D.  M. ;  Bresnick,  T.  A.;  and  Kuskey,  K.  P. 

The  Ranking  Process  for  the  Development  of 
the  Army's  POM  82  Submission;  Summary  and 
Critiqu"^  Interim  Technical  Report  PR 
60-11-153.  McLean,  Virginia:  Decisions  and 
Designs,  Inc.,  May  1980. 
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Buede,  D.  M.,  and  Kuskey,  K.  P.  U.S.  Marine 
Corps  POM-81  Program  Benefits  of  Competitive 
PrograniFI  Interim  Report  PR  75-11-92. 
McLean,  Virginia:  Decisions  and  Designs, 
Inc.,  June  1979. 


Buede,  D.  M.;  Kuskey,  K.  P.;  Ragland,  J.  E.;  and 
Peterson',  C.  B.  Use  of  Decision  Analysis  for 
the  1981-1985  U.S.  Marine  Corps  Program  Objec¬ 
tives  Memorandum  Preparation.  Final  Report 
PR  79-12-9i.  McLean,  Virginia:  Decisions 
and  Designs,  Inc.,  September  1979. 

Buede,  D.  M.,  and  Kuskey,  K.  P,  An  Appraisal 

of  the  Army*s  New  Approach  to  Program  Prior iti 
zation  for  POM  82-86.  Final  Technical  Report 
PR  86-14-153.  McLean,  Virginia:  Decisions 
and  Designs,  Inc.,  October  1980. 

Buede,  D.  M. ,  and  Peterson,  C.  R.  An  Application 
of  Cost-Benefit  Analysis  to  the  USMC  Prograir 
Objectives  Memorandum  (PO^^  Technical 
Report  TR  77-8-72.  McLean,  Virginia;  Deci¬ 
sions  and  Designs,  Inc.,  September  1977, 

Buede,  D.  M. ,  and  Ragland,  J.  E.  Cost-Benefit 
Analysis  Applied  to  the  Program  Objectives 
Memorandum  (POM) .  Technical  Report  TR  78-9- 
72 .  McLean,  Virginia:  Decisions  and  Designs, 
Inc.,  November  1978. 

Feuerwerger,  P.  H.;  Ulvila,  J.  W. ;  and  Buede,  D. 
Program  Prioritization  Guide.  Final  Technical 
Report  PR  79-17-$5.13.  McLean,  Virginia: 
Decisions  and  Designs,  Inc.,  October  1979. 

Gulick,  R.  M.;  Kuskey,  K.  P.;  and  Buede,  D.  M. 
i  An  Analysis  of  Investment  Opportunities  for 

Night  and  In-Weather  Tactical  Air  Operations. 
Final  Technical  Report  PR  79-2^-309.  McLean , 
Virginia:  Decisions  and  Designs,  Inc.,  Decem¬ 

ber  1979. 

Kraft,  Robert  N.;  Buede,  Dennis  M.;  and  Martin, 
Anne  W.  Resource  Allocation  for  OAST/Space 
Growth  Projects.  Final  Report  PR  81-3-163. 
McLean,  Virginia:  Decisions  and  Designs,  Inc. 
January  1981. 

Kuskey,  K.  P.  Program  Cost  Estimation  for  POM 
Development.  Interim  Report  PR  79-1-92. 
McLean,  Virginia:  Decisions  and  Designs, 

Inc. .  March  1979. 


Kuskey,  K.  P.;  Bresnick,  T.  A.;  Buede,  D.  M. ; 
and  Waslo’s  K.  A.  U.S.  Marine  Corps  POM- 82; 
Review  of  Changes  to  the  POM  Development 
Process^  Final  Technical  Report  PR  80-18-150 . 
McLean,  Virginia:  Decisions  and  Designs,  Inc., 
November  1980. 

Waslov,  K.  A.  Prioritization  and  Cost  Benefit 
Analysis  of  MERADCOM  Science  and  Technology 
Projects;  Summary  and  Critique.  Final  Tecn- 
nical  Report  PR  80-20-10.59.  ^McLean,  Virginia: 
Decisions  and  Designs,  Inc.,  September  1980. 

Waslov,  K.  A.,  and  Kuskey,  K.  P.  U.S.  Marine 
Corps  POM-82  Mission  Area  Analysis.  Interim 
Report  PR  80-16-150.  McLean ,  Virginia : 
Decisions  and  Designs,  Inc.,  July  1980. 


2.4.3  Basic  theory  and  methodology  - 


Fisher,  G.  H.  Cost  Considerations  in  Systems 
Analysis .  New  York:  American  Elsevier 
Publishing  Company,  Inc.,  1971. 

Hinrichs,  H.  H.,  and  Taylor,  G.  M. ,  eds.  Pro¬ 
gram.  Budgeting  and  Benefit-Cost  Analysis; 

Cases,  Text,  an3  Readings.  Pacific  Palisades , 
California:  Goodyear  Publishing  Co.,  Inc., 

1969  . 

Keeney,  R.  L.,  and  Raiffa,  H.  Decisions  with 
Multiple  Objectives;  Preferences  and  Value 
Trade-offs.  New  York:  John  Wiley  &  Sons, 

1976. 

Krantz,  D.  H.;  Luce,  R.  D.;  Suppes,  P.;  and 
Tversky,  A.  Foundations  of  Measurement, 

Vol  I,  Additive  and  Polynomial  Representations. 
New  YorJn  Academic  Press,  1971 . 

Luce,  R.  D.,  "Conjoint  Measurement;  A  Brief  Survey," 
Chapter  6  of  David  E.  Bell,  Ralph  L.  Keeney, 
and  Howard  Raiffa,  eds.  Conflicting  Objectives 
in  Decisions.  New  York:  John  Wiley  &  Sons, 

1977. 


2.4.4  POM  Data  Base  Management  System.  - 


Esoda,  R.  M.  POM-DBMS  Program  Maintenance 
Manual .  UM  81-2-158.  McLean,  Virginia : 
Decisions  and  Designs,  Inc.,  1981. 
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Waslov,  K.  A.;  Kuskey,  K.  P.  POM-DBMS  Users 
Manual.  UM  81-1-158.  McLean,  Virginia: 
Decisions  and  Designs,  Inc.,  1981. 
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3.0  DECISION-ANALYTIC  STRATEGY  AND  GUIDELINES 


Within  the  Marines'  approach  to  programming,  prioriti¬ 
zation  is  the  key  feature  that  affects  all  others;  (1)  the 
process,  methods,  and  guidelines  established  for  prioritiza¬ 
tion  naturally  create  incentives  that  determine  the  character 
of  the  sponsors'  initiatives;  and  (2)  the  prioritized  list 
of  programs  forms  the  basic  guide  for  constructing  the  POM 
submission.  Decision-analytic  methods  are  used  primarily 
by  the  MC  staff  to  support  the  prioritization  stage  of 
programming,  including: 

(1)  the  design  of  prioritization  procedures; 

(2)  the  proper  formulation  of  program  initiatives; 

(3)  the  collection  of  professional  military  judgments 
about  the  importance  of  initiatives; 

(4)  the  integration  of  these  judgments;  and 

(5)  the  prioritization  of  programs  using  judgments  of 
worth  tempered  by  considerations  of  program  costs. 

This  section  presents  the  Marines'  methodological 
strategy  and  guidelines  for  conducting  the  prioritization 
stage  of  programming.  As  described  already,  this  stage  is 
applied  several  times  during  POM  development--f irst  to 
military  manpower  initiatives,  then  to  civilian  manpower 
initiatives,  then  to  the  procurement  and  O&M  initiatives. 

It  may  also  be  applied  selectively  to  the  MC ' s  RDT&E  initia¬ 
tives  and  its  "blue-dollar"  aviation  initiatives.  The 
features  of  prioritization  methodology  described  in  this 
section  apply  equally  to  all  these  different  tasks. 


The  section  begins  with  some  background  on  the  various 
approaches  to  program  prioritiz^cion.  It  then  describes  the 
MC's  approach  and  discusses  the  unique  features  that  distin¬ 
guish  it  from  the  other  services.  Finally,  it  describes  the 
MC's  present  guidelines  for  carrying  out  the  approach.  Sec¬ 
tion  4.0  presents  the  detailed  methods  used  to  implement 
these  guidelines. 

3 . 1  Background  on  Prioritization  in  DoD 

A  POM  is  a  proposal  to  spend  money  on  specific  programs 
and  forces.  Through  the  minimum,  basic  and  enhanced  levels, 
it  identifies  the  relative  importance  of  programs  to  guide 
DoD  decision  makers  as  they  allocate  scarce  resources  across 
the  military  services.  Obviously,  the  same  set  of  programs 
can  be  packaged  and  prioritized  several  ways  by  a  service. 

The  prioritization  problem  is  to  build  the  best  POM  possible 
within  normal  constraints  of  time  and  effort. 

Ail  the  services  approach  the  POM  as  follows:  (1)  de¬ 
velop  a  prioritized  list  of  programs  that  shows  the  service's 
order  of  preference  among  them  individually  or  in  large 
groups  (i.e.,  m.inimum,  basic,  enhanced);  (2)  package  the 
large  list  of  programs  into  a  DoD-required ,  smaller  number 
of  packages,  using  the  priority  list  as  a  guide.  In  all  the 
services,  the  first  step  is  performed  through  systematic 
staff  action.  The  second  is  always  a  matter  of  ad  hoc 
programming  judgment  since  there  is  not  a  generally  accepted 
logic  for  consolidating  programs  into  packages.  (This  is 
not  to  say  such  logic  cannot  be  developed,  just  that  each 
service  uses  a  unique  logic.)  The  overall  logic  of  POM  de¬ 
velopment  is  a  belief  that  if  a  good  priority  list  can  be 
developed,  then  the  packaging  of  programs  into  a  POM  can  be 
accomplished  well  by  experienced  programmers. 


All  the  services  approach  the  prioritization  of  prograiris 
through  hierarchical  decomposition;  a  prioritized  list  is 
formed  by  (1)  assembling  several  independent  lists  for 
functionally  different  areas,  and  (2)  merging  the  independent 
lists.  This  hierarchy  may  extend  to  several  levels  as  lists 
are  made  from  lists  made  from  lists.  There  are  two  approaches 
to  this  hierarchical  prioritization.  The  first  is  exempli¬ 
fied  by  OSD,  the  SECNAV,  and  the  CKO;  the  second,  by  the 
Army,  Air  Force,  and  Marine  Corps. 

In  the  first  approach,  called  "multi-level  programjning , " 
the  independent  lists  are  designed  by  independent  functional 
sponsors  to  meet  centrally  determined,  multi-level  fiscal 
and  programmatic  guidance.  Thus,  OSD's  CG  specifies  fiscal 
constraints  to  the  services  for  minimum-,  basic-,  and  enhancecl- 
prograr  "levels."  The  constraints  are  in  dollars  of  total 
obligational  authority  for  each  of  the  five  POK  years. 

Along  with  the  fiscal  guidance,  the  CG  specifies  program¬ 
matic  goals  that  the  services  should  meet  at  each  level. 

Having  received  the  fiscal  and  programmatic  guidance,  the 
services  determiine  their  own  programiS  for  minimum,  basic  and 
enhanced  levels.  While  their  placement  of  programs  in  the 
levels  is  subject  to  review,  it  is  basically  their  responsi¬ 
bility  to  design  their  own  programs  from  the  guidance. 

The  minimum,  basic,  and  enhanced  levels  of  the  CKO's 
POM  submission  are  formed  by  combining  minimum-level  programs 
submitted  by  the  CKO's  resource  sponsors,  then  their  basic- 
level  programs,  and  then  their  enhanced-level  programs. 

Since  each  sponsor's  submission  meets  CKO  fiscal  guidance 
for  each  level,  the  CKO's  input  to  the  DoK  POM  meets  SECKAV 
fiscal  guidance.  This  is  the  simple  picture,  of  course.  In 
practice,  the  CKO's  programmers  move  some  of  each  sponsor's 
programs  from  one  level  to  another.  In  this  regard,  we  may 
note  that  the  OSD/SECKAV/CKO  method  of  prioritization  has  no 
formal  provision  to  rove  programs  from  one  level  to  another 
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based  on  service  or  sponsor  priorities.  The  services  or 
sponsors  are  not  required  to  prioritize  programs  within  each 
level,  though  they  often  do  put  them  in  approximate  order. 
Thus,  it  is  not  possible  to  determine  from  a  service's 
submission  exactly  what  its  priorities  are  among  programs 
and  program  packages. 

The  OSD  and  Navy  methods  extend  considerable  programming 
authority  to  the  independent  program  sponsors  In  that  each 
sponsor  selects  the  program  level  for  each  of  its  programs. 

If  each  has  followed  its  guidance,  then  OSD  and  the  Navy  can 
produce  a  total  program  without  further  analysis  merely  by 
adding  the  sponsors'  programs  ^or  each  level.  The  true 
picture  is  much  more  complex,  but  this  simple  one  is  adequate 
to  contrast  the  OSD/Navy  method  with  the  Army/Air  Force/ 
Marine  Corps  method. 

Hierarchical  prioritization  is  handled  differently  from 
the  OSD  and  the  Navy  by  the  Army,  the  Air  Force,  and  the 
Marine  Corps,  For  these  services,  when  independent  sponsors 
are  requested  to  propose  program  initatives,  they  are  not 
given  fiscal  guidance  or  programir.atic  guidance  tied  to 
funding  level.  That  is,  their  proposals  are  not  fiscally 
and  programmatically  constrained.  Rather,  the  sponsors  are 
encouraged  through  guidance  to  propose  initiatives  that 
effectively  satisfy  needs.  This  means  practically  that 

(1)  the  programming  headquarters  receives  a  larger  number  of 
initiatives  than  it  can  program  even  at  the  enhanced  level; 

(2)  the  programmers  rather  than  the  sponsors  assign  initia¬ 
tives  to  funding  levels;  and  (3)  the  programmers  rather  than 
the  sponsors  implement  the  OSD  programmatic  guidance  relative 
to  funding  level. 

Once  the  Army,  Air  Force,  and  MC  receive  program  initia¬ 
tives  from  sponsors,  they  proceed  by  different  methods  to 
prioritize  them.  The  Army  for  instance,  will  assign  each  to 
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a  mission  area  proponent  (MAP) ,  and  the  MAPs  will  screen, 
evaluate,  and  prioriti2e  them.  Then  the  separate  prioritized 
lists  from  the  MAPs  will  be  merged  by  a  central  programming 
office.  The  Marines,  on  the  other  hand,  assist  the  sponsors 
to  prioritize  their  own  initiatives.  The  prioritized  lists 
are  merged  by  a  central  MC  staff.  In  each  case,  a  complete 
order  (in  the  mathematical  sense)  is  determined  separately 
for  each  sponsor  or  proponent's  initiatives.  Then  a  complete 
order  is  determined  for  all  initiatives  by  "merging"  the 
sponsors'  or  proponents'  lists.  By  "merging,"  we  mean  that 
each  sponsor  or  proponent’s  initiatives  in  the  final  list 
are  in  the  same  order  they  were  in  the  separate  lists,  with 
occasional  exceptions.  To  coin  a  term,  we  shall  call  this 
method  of  prioritization  "priority  programming"  to  distin¬ 
guish  it  from,  multi-level  programming.  It  differs  from  the 
latter  by  building  a  complete  priority  order  of  programs 
rather  than  merely  assigning  each  program,  to  one  of  three 
fiscally  constrained  levels. 

The  simplest  method  for  merging  several  prioritized 
lists  requires  superhuman  mental  skills  in  military  applica¬ 
tions.  This  method  is  called  the  "blue  card"  method  (BCM) 
because  w^  first  saw  it  carried  out  with  blue  3x5  file  cards. 
Several  stacks  of  cards  were  made,  one  for  each  sponsor,  and 
each  card  had  the  name  of  a  program,  initiative. 

Each  sponsor  put  his  cards  in  order  of  program  priority 
with  his  best  program  on  top.  The  stacks  of  prioritized 
cards  were  put  on  a  table,  and  several  colonels  with  no 
responsibility  to  any  program  sponsor  picked  one  card  at  a 
time  from  the  stacks  to  form  one  single  stack.  The  new 
stack  had  all  the  cards  in  it,  and  they  were  in  the  priority 
order  that  the  group  had  determined.  It  had  several  hundred 
cards  and  represented  several  billion  dollars. 
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While  the  BCM  sounds  plausible,  it  turns  out  to  have 
serious  drawbacks.  The  basic  problem  is  that  each  of  the 
several  hundred  programs  to  be  prioritized  must  be  examined 
at  least  once  by  the  group,  and  each  program  must  be  compared 
to  very  different  programs  from  other  sponsors.  It  is  im¬ 
possible  for  a  reasonably  sized  group  (5-20  people)  to  have 
both  an  in-depth  knowledge  of  each  of  several  hundred  programs 
as  well  as  the  broad,  unbiased  service  viewpoint  needed  to 
merge  all  the  progreims  during  the  time-constrained  POM 
development.  To  our  knowledge,  no  one  who  has  tried  this 
process  has  continued  to  use  it. 

These  observations  about  the  BCM  would  not  apply  if 
there  were  only  a  few  initiatives  and  only  a  very  few  broad 
objectives  to  be  met  with  the  initiatives.  The  BCM  might 
work  well  for  a  small  organization  with  a  small  budget.  We 
believe  it  will  seldom  work  well  for  a  large  organization 
such  as  the  MC.  Although  it  will  correctly  identify  the 
best  and  the  worst  programs,  lack  of  information  or  lack  of 
a  sense  of  priorities  will  tend  to  randomize  the  priorities 
of  the  intermediate  programs  that  are  the  subject  of  diffi¬ 
cult  decision  making.  The  MC ' s  method  overcomes  these 
difficulties,  as  explained  below. 

3.2  The  MC ' s  Method  of  Prioritization 


The  MC ' s  approach  to  priority  programming  is  an  inno¬ 
vation  that  puts  it  ahead  of  the  other  services  in  its 
ability  to  bring  the  collective  expertise  of  its  staff  to 
bear  on  program  development.  This  section  discusses  the 
essential  elements  of  their  method.  These  are  (1)  dis¬ 
ciplined  formulation  of  initiatives;  (2)  hierarchical, 
quantitative  evaluation  of  initiatives;  and  (3)  cost- 
effectiveness  analysis.  Subsequent  sections  describe  the 
prioritization  ground  rules  and  the  detailed  procedures  for 
carrying  out  the  method. 
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3.2.1  Formulation  of  initiatives  -  A  deliberate  approach 
to  the  formulation  of  project--:  is  a  key  element  of  priority 
programming  systems.  The  process  of  prioritization  is 
frustrated  if  the  items  being  evaluated  do  not  fit  the 
process;  their  priorities  become  obscured.  For  instance,  if 
items  are  formulated  strictly  within  appropriation  lines 
(because  they  can  be  managed  more  easily  by  appropriation 
sponsors,  if  executed) ,  they  tend  to  be  impossible  to 
evaluate  singly  in  a  prioritization  system  that  is  focused 
on  mission  capabilities.  This  is  because  a  single,  execut¬ 
able  mission  capability  almost  always  involves  several  ap¬ 
propriations  (to  cover  investment,  transportation,  training, 
operations,  maintenance,  and  manpower) ,  and  no  one  of  these 
should  be  evaluated  singly,  any  more  than  "left  shoe"  should 
be  evaluated  against  "right  shoe"  to  fulfill  the  mission 
"walking. " 

At  the  MC  headquarters,  criteria  for  formulating 
initiatives  are  first  communicated  to  the  sponsors,  and  then 
they  are  used  to  validate  the  sponsors'  initiatives  before 
they  can  compete  for  a  place  in  the  POM.  No  institution 
should  expect  success  in  applying  the  MC ' s  method  (or  any 
other) ,  if  it  is  unable  to  discipline  project  formulation 
similarly.  Without  such  control  the  system  invariably 
breaks  down. 

The  staff's  MC  criteria  for  formulating  initia¬ 
tives  are  that  they  should  be  (1)  executable;  (2)  incremental; 
(3)  mission-oriented;  (4)  independent;  and  (5)  significant. 
These  criteria  overlap  to  some  degree,  but  they  provide  five 
different  ways  of  validating  initiatives  as  POM  competitors. 

An  executable  initiative  contains  within  it  all 
the  parts  necessary  to  its  implementation,  and  each  part  is 
capable  of  implementation.  An  initiative  to  "buy  trucks"  is 
not  executable  if  there  are  no  trucks  to  buy.  An  initiative 


to  "equip  a  division  with  M198  howitzers"  is  not  executable 
if  it  does  not  include  prime  movers  for  the  howitzers,  since 
the  howitzers  are  useless  if  they  cannot  be  moved. 

An  initiative  is  incremental  if  (1)  it  has 
relatively  small  though  significant  importance  when  compared 
to  the  MC's  total  capability,  and  (2)  it  is  designed  as  a 
change  to  the  status  quo  in  which  something  new  is  added  and 
something  current  may  be  deleted.  As  to  the  first  point, 
although  the  POM  includes  large  changes  when  they  occur 
(e.g.,  the  rapid  deployment  force  in  POM  82),  it  is  several 
hundred  smaller  changes  that  are  the  real  subject  of  POM 
prioritization  from  year  to  year.  The  larger  issues  are 
decided  separately  and  do  not  compete  in  the  prioritization. 
However,  it  is  not  always  clear  to  everyone  whether  an  issue 
is  so  large  that  it  should  be  decided  separately.  An  example 
of  an  issue  in  this  gray  area  between  large  and  small  in 
the  MC's  POM  82  was  the  replacement  of  unserviceable, 
obsolete  computers  vital  to  the  MC’s  tactical  and  general 
logistics  and  administrative  systems.  Many  among  the  MC 
staff  were  completely  unaware  of  the  degree  to  which  all 
aspects  of  MC  operations  depend  on  these  computers.  The 
Marines  leave  such  an  issue  in  the  prioritization  process 
with  the  understanding  that  its  status  as  a  competitor  may 
be  changed  later. 

The  concept  of  an  "incremental"  initiative 
concerns  more  than  size,  however.  Primarily,  the  word 
"incremental"  puts  emphasis  upon  the  full  difference  in  MC 
operations  between  having  and  not  having  an  initiative.  A 
validly  incremental  initiative  contains  within  it  not  only 
something  new  but  also  the  proper  adjustments  to  current  and 
future  operations  that  should  accompany  it.  An  initiative 
to  buy  a  new  type  of  truck  for  instance,  should  include 
explicit  plans  to  decrease  purchases  and  maintenance  of  the 
current  type  of  truck.  Otherwise,  it  is  not  incremental, 
and  it  is  not  formulated  properly. 
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For  the  MC,  a  properly  formulated  initiative 
must  be  "mission-oriented"  as  well  as  executable  and  incre¬ 
mental.  The  MC  first  used  mission  areas  as  the  basis  for 
POM  development  for  POM  81.  Seventeen  mission  areas  were 
defined.  This  concept  was  successful,  and  it  was  used  again 
for  POM  82,  which  had  26  mission  areas.  The  39  mission 
areas  planned  for  POM  83  were  shown  above  in  Figure  2-1. 

A  mission-oriented  initiative  is  defined  and 
evaluated  in  term.s  of  one  specific  mission  capability  that 
it  will  provide  the  Marine  Corps  (though  it  is  usually  named 
for  the  chief  item  it  provides) .  Given  its  mission  orienta¬ 
tion,  the  MC ' s  evaluation  of  initiatives  is  outcome-oriented 
rather  than  input-,  process-,  or  institution-oriented.  That 
is,  it  lets  the  Marines  build  a  program  based  on  im, proving 
explicit  military  capabilities  rather  than  merely  improving 
its  base  of  resources  or  its  management  of  personnel  and 
resources.  Contrary  to  the  mission-area  orientation,  a 
tendency  of  program  designers  is  to  define  programs  along 
appropriation  lines.  In  this  case  (1)  no  initiative  provides 
a  complete  mission  capability,  only  parts?  and  (2)  an  initia¬ 
tive  may  combine  parts  of  several  different  mission  capabili¬ 
ties  because  they  can  be  grouped  logically  in  the  same 
appropriation  (or  budget)  line.  Both  these  situations  cause 
confusion  in  a  mission-oriented  analysis. 

The  next  criterion  is  that  initiatives  should  be 
-.ndependent.  This  means  that  ideally, 

o  the  military  capability  provided  by  any  one  ini¬ 
tiative  should  not  depend  strongly  on  the  capa¬ 
bilities  that  would  be  provided  by  other  initia¬ 
tives  if  they  were  funded;  and 

o  the  resource  requirements  of  an  initiative  should 
not  depend  strongly  on  the  funding  of  any  other 
initiative. 
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It  requires  considerable  effort  to  formulate 
independent  progrcuns.  However,  the  fact  that  the  programs 
are  all  increments  to  the  core  of  MC  activities  that  is  not 
being  prioritized  (Section  2. 3. 2. 2)  makes  independence 
easier  to  achieve  than  one  might  expect.  The  tendency  to  be 
guarded  against  is  to  formulate  initiatives  so  narrowly  that 
they  provide  only  part  of  a  mission-oriented  capability,  in 
which  case  their  usefulness  strongly  depends  on  the  funding 
of  the  other  parts. 

Apart  from  such  tendencies,  there  is  always 
difficulty  in  formulating  logical  independent  initiatives 
for  the  missions  of  communications,  command  and  control, 
automatic  data  processing  systems,  and  base  operations. 

These  missions  require  networks  of  equipment,  personnel,  or 
operations.  Often  an  executable  mission-oriented  initiative 
can  only  be  formulated  as  a  combination  of  several  integrated 
changes  to  the  network.  Rarely  will  such  initiatives  be 
independent  since  they  represent  different  ways  of  integrating 
the  elements  of  the  same  network.  A  systematic  strategy  for 
formulating  such  initiatives  is  outlined  in  Section  4.3.3. 

It  makes  possible  their  prioritization  along  with  the  rest 
of  the  POM  competitors. 

Finally,  a  POM  initiative  should  significant 
to  compete.  The  MC  staff  uses  the  rule  of  thumb  that  current 
capability  programs  s.*ould  expend  resources  of  at  least  $1M 
in  five  years  to  compete  with  new  initiatives.  This  assures 
that  the  current  capability  will  have  adequate  scope  to 
merit  careful  evaluation  along  with  all  other  items.  This 
criterion  is  not  held  rigidly,  but  it  is  the  assumed  rule 
and  is  enforced  with  few  exceptions.  No  rule  of  thumb  is 
applied  to  new  initiatives  themselves,  but  small  initiatives 
are  generally  included  in  the  POM  as  "fillers"  rather  than 
as  true  competitors  for  resources. 
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In  svunmary,  the  MC  staff  requires  each  initia¬ 
tive  to  be  executable,  incremental,  mission-oriented, 
independent,  and  significant.  When  the  initiatives  meet 
these  requirements,  they  can  be  prioritized  readily  in  the 
MC's  priority-programming  system.  When  they  do  not  substan¬ 
tially  meet  the  requirements,  they  only  confound  the  pro¬ 
gramming  system.  The  MC  staff  applies  these  requirements, 
therefore,  to  validate  initiatives  before  they  compete  for  a 
place  in  the  POM. 

3.2.2  Hierarchical  evaluation  of  initiatives  -  A  dif¬ 
ficult  feature  of  priority  programming  for  large  organizations 
is  the  great  diversity  eimong  their  initiatives.  For  instance, 
the  MC's  POM  83  submission  will  likely  contain  such  dissimilar 
initiatives  as  the  construction  of  gymnasiums,  product 
improvement  of  HAWK  missiles,  accelerated  procurement  of 
howitzer  ammunition,  replacem.ent  of  administrative  data 
processing  systems,  a  new  computer  system  to  aid  the  planning 
of  officers'  careers,  the  overhaul  of  tank-recovery  vehicles, 
and  a  field  water  purification  unit.  To  set  priorities  well 
among  several  hundred  such  initiatives  is  very  difficult  at 
best,  if  not  imipossible,  because  of  their  dissimilarity. 

The  MC  staff's  mission-oriented  programming 
achieves  many  purposes,  and  one  of  the  most  important  is 
that  it  groups  similar  initiatives  together  under  each 
mission.  This  grouping  makes  it  practical  for  officers  who 
are  specialists  in  the  mission  areas  to  assist  the  MC's 
programmers  to  quickly  and  accurately  prioritize  the  initia¬ 
tives  within  their  specialties  without  being  asked  to  learn 
other  specialties.  Maximum  use  is  made  of  their  special 
expertise. 

The  prioritization  of  similar  initiatives  within 
mission  areas  by  specialists  solves  part  of  the  difficulty 
of  evaluating  very  diverse  initiatives,  but  only  part.  The 


separate  lists  of  well-prioritized  initiatives  must  yet  be 
merged,  for  no  automatic  merge  is  implicit  in  the  separate 
lists.  (In  fact,  the  MC's  separate  lists  can  be  merged  in 
literally  billions  of  ways.) 

The  MC's  strategy  for  merging  the  separate 
mission  lists  is  based  on  the  idea  of  "sampling"  initiatives 
from  each  list,  determining  the  priorities  of  the  samples, 
and  using  these  priorities  to  determine  what  all  other 
initiatives'  priorities  should  be.  To  accomplish  this,  it 
builds  on  concepts  of  mathematical  psychology  for  measuring 
the  importance  of  projects.  The  basic  process  is  as  follows*. 

(1)  The  within-mission  evaluations  of  initiatives  are 
performed  in  such  a  way  that  analysts  can  construct 
an  objective  numerical  representation  of  the  rela¬ 
tive  subjective  importances  of  the  initiatives 
within  each  mission,  but  not  across  missions.  For 
each  mission,  bigger  numbers  signify  more  important 
initiatives;  and  bigger  sums  of  these  numbers  sig¬ 
nify  more  important  packages  of  initiatives.  The 
numbers  are  called  "mission  benefits." 

(2)  Three  sample  initiatives  are  chosen  to  represent 
each  mission  area.  All  sample  initiatives  are 
briefed  by  their  sponsors  to  a  program  evaluation 
group  (PEG) .  The  PEG  itself  is  composed  of  offi¬ 
cers  who  know  the  relative  importances  of  the  mis¬ 
sion  areas  to  the  Marines  because  of  their  field 
and  staff  experience.  They  generally  do  not  know 
the  details  of  the  sampled  programs  until  they  are 
briefed. 

(3)  The  PEG  evaluates  all  the  sample  initiatives  as 
briefed  to  them..  Analysts  working  with  them  de¬ 
velop  a  numerical  representation  of  the  importance 
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of  each  initiative  relative  to  one  another  as  seen 
by  the  PEG.  These  numbers  are  called  "cross¬ 
mission  benefits." 

(4)  The  relative  importances  of  the  scunples  from  a 
mission  area  may  be  perceived  differently  by  the 
PEG  and  the  mission-area  experts.  If  so,  the 
ratios  of  the  initiatives'  mission  benefits  for 
each  sample  of  three  will  differ  from  the  ratios 
of  their  cross-mission  benefits.  When  the  differ¬ 
ence  is  large,  the  PEG  meets  with  the  mission  ex¬ 
perts  to  reconcile  the  differences.  In  this 
exchange,  the  PEG’S  final  perception  of  the  ini¬ 
tiatives  prevails.  If  necessary,  all  of  the  mis¬ 
sion  benefits  for  a  mission  area  are  then  adjusted 
mathematically  to  be  consistent  with  the  PEG'S 
final  cross-mission  benefits  for  the  sample  items 
in  the  mission  area. 

(5)  The  peg's  final  cross-mission  benefits  are  used  to 
determine  a  single  weighting  factor  for  each  mission 
area.  The  "overall  benefit"  of  each  initiative  is 
calculated  by  multiplying  each  initiative's 
adjusted  mission  benefit  by  the  weighting  factor 
for  its  mission  area.  All  initiatives  are  then 
listed  in  order  of  overall  benefit,  and  this  list 

is  the  final  product,  subject  to  "fine  tuning." 

This  process  differs  strongly  from  most  prioriti¬ 
zations  because  the  Marines'  mission  and  PEG  experts  not 
only  put  items  in  priority  order,  but  establish  a  means  for 
comparing  any  combination  of  items  to  any  other  combination 
of  the  items,  i.e.,  for  determining  which  combination  would 
be  more  valuable.  This  requires  a  much  more  thorough 
analysis  of  initiatives  than  the  task  of  merely  putting  them 
in  order,  but  this  is  necessary  if  separate  lists  are  to  be 


merged  by  taking  samples  from  each.  The  numerical  representa¬ 
tion  of  importance  is  built  from  this  more  thorough  analysis. 
As  an  important  side  benefit,  the  numbers  provide  an  objective 
means  for  the  mission  experts  and  PEG  to  communicate  about 
the  relative  importance  they  place  on  initiatives:  from  a 
simple  ordered  list,  one  cannot  learn  whether  (1)  the  first 
item  and  last  item  are  nearly  as  important  as  one  another, 
or  (2)  the  first  item  is  more  important  than  all  other  items 
combined.  The  mission  benefits  and  overall  benefits  communi¬ 
cate  immediately  how  much  importance  each  item  has  (1)  within 
its  miission,  and  (2)  to  the  MC  as  a  whole.  These  improve 
communications  among  the  ^fC  staff,  as  well  as  provide  the 
basis  for  merging  the  mission  areas. 

Practically,  the  Marines  have  too  many  miission 
areas  for  the  PEG  to  realistically  evaluate  samples  from  all 
of  them  at  once.  Three  samples  from  38  mission  areas  would 
require  the  PEG  to  hear  117  briefings  and  remember  them 
while  prioritizing  the  initiatives--a  super-human  task. 

The  MC  strategy  is  to  merge  the  mission  areas  hierarchically: 
nine  clusters  containing  one  to  ten  missions  are  separately 
merged,  each  with  the  process  described  above;  then,  three 
samiples  from,  each  cluster's  merged  list  are  taken  and  used 
to  merge  the  cluster  lists,  using  the  same  process.  This 
keeps  the  human  tasks  of  evaluation  within  credible  bounds. 

In  Figure  2-1,  which  lists  the  38  mission  areas  the  Marines 
plan  to  use  for  POM  83,  the  bold  horizontal  lines  mark  the 
planned  clusters  of  mission  areas. 

The  psychological  motivations  for  the  hierarchi¬ 
cal  prioritization  procedure  are  (1)  to  foster  more  objective 
and  effective  communication  about  subjective  matters  aimong 
professional  military  experts,  and  (2)  to  improve  the  experts' 
judgments  by  reducing  in  so  far  as  possible,  "apples  versus 
oranges"  evaluations  in  the  process.  By  contrast,  the  blue- 
card  method  (Section  3.1)  requires  a  maximum  number  of 
apples  versus  oranges  judgments.  The  MC ' s  procedure  deals 


successfully  with  the  diversity  among  their  initiatives, 
leading  to  prioritizations  that  have  broad  support  among  the 
MC  staff. 


3.2.3  Cost-effectiveness  analysis  -  Once  the  overall 
benefits  are  quantified,  diverse  initiatives  are  readily 
compared  in  terms  of  both  importance  and  cost.  Often,  two 
or  three  diverse  programs  of  moderate  importance  are  collec¬ 
tively  more  important  but  less  costly  than  some  other  highly 
important  program.  Such  analyses  can  be  made  from  the 
overall  benefits  and  costs  of  the  programs  without  further 
professional  military  judgment.  Thus,  these  situations  can 
readily  be  analyzed  by  the  programming  staff  during  POM 
preparation.  The  question  is,  what  use  should  be  made  of 
such  analysis? 

As  mentioned  in  Section  2.3,  the  Marines  are  not 
merely  allocating  resources  with  their  POM,  they  are  also 
attempting  to  secure  resources.  If  they  program  three 
moderately  important  initiatives  rather  than  one  highly 
important  one,  the  wisdom,  of  this  may  not  be  seen  by  OSD. 

If  it  is  not  seen,  the  MC ' s  program  will  be  perceived  as 
weaker  rather  than  stronger.  Clearly,  great  care  must  be 
exercised  in  making  cost-effective  program  substitutions. 

They  might  lead  to  smaller  overall  funding  and  a  weaker 
capability  if  they  are  not  perceived  correctly.  For  this 
reason,  the  MC  staff  makes  such  substitutions  only  by  excep¬ 
tion.  On  the  other  hand,  cost-effectiveness  analysis  is 
performed  routinely  by  the  MC  staff  to  look  for  worthwhile 
substitutions . 

The  more  mundane  roles  of  cost-effectiveness 
analysis  are: 

(1)  to  identify  important  programs  that  have  been 

"padded"  with  unimportant  but  expensive  components, 
which  makes  them  look  poorly  cost-effective;  and 
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(2)  to  identify  moderately  important  programs  that 
have  failed  to  include  expensive  but  essential 
components,  which  makes  them  look  highly  cost- 
effective  . 

The  Marines'  cost-effectiveness  analysis  is 
different  from  the  traditional  form  because  "effectiveness" 
is  not  measured  in  objective  terms.  It  is  a  measure  of  the 
subjectively  perceived  military  importance  of  initiatives. 

In  theory,  a  POM  based  wholly  on  the  cost-effectiveness  of 
initiatives  would  achieve  most  quickly  the  MC ' s  collective 
sense  of  program  priorities,  since  that  is  what  the  "overall 
benefits"  measure.  An  explicit  premise  of  the  MC  staff's 
approach  is  that  the  MC  ultimately  depends  for  programming 
on  the  professional  expertise  and  sense  of  priorities  of  its 
senior  managers  and  staff  experts,  not  on  objective  measures 
of  effectiveness.  Their  approach  elicits  and  uses  this 
expertise  and  sense  of  priorities  to  build  the  POM. 

3.2.4  Summary  -  This  section  has  described  the  basic 
elements  of  the  MC ' s  prioritization:  (1)  disciplined  formu¬ 
lation  of  initiatives;  (2)  hierarchical,  quantitative  evalu¬ 
ation  of  initiatives;  and  (3)  cost-effectiveness  analysis. 
The  following  section  describes  the  prioritization  ground 
rules  in  effect  for  POM  83;  these  rules  may  change  in  the 
future.  Section  4.0  describes  detailed  procedures  for 
carrying  out  the  method. 

3.3  Staff  Guidelines 


The  Marines'  method  of  prioritization  requires  a  coordi¬ 
nated  effort  by  the  MC  staff.  This  section  describes  the 
assignment  of  responsibility  among  the  staff  for  the  various 
judgments  necessary  to  carry  out  the  method.  This  assignment 
of  responsibility  was  current  for  the  POM  83  effort. 
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3.3.1  General 


f 


(1)  Sponsors'  initiatives  must  be  formulated  properly 
to  compete  for  funding.  Validation  procedures 
include  obtaining  DC/S-level  concurrence  from  all 
sponsors  that  each  initiative  is  executable  and 
incremental.  The  PWG  will  establish  that  they  are 
mission-oriented,  independent,  and  significant. 

(2)  Initiatives  must  be  prioritized  within  mission 
areas;  each  initiative  is  assigned  to  just  one 
mission  area.  Upon  request  from  DC/S  R&P  RPP, 
sponsors  assign  representatives  to  provide  miilitary 
expertise  for  mission-area  prioritizations;  then 
prioritization  mergings  are  conducted  by  RPP  to 
determine  priorities  and  mission  benefits  for 
initiatives.  RPP  has  no  role  in  setting  priorities 
other  than  to  provide  methods  and  procedures.  As 
part  of  their  work,  the  sponsors'  representatives 
must  develop  statements  of  mission-area  capabili¬ 
ties  and  deficiencies. 

(3)  When  the  same  sponsor  is  providing  expertise  for 
two  or  more  mission  areas,  then--if  practicable-- 
the  sponsor  also  provides  representatives  to  work 
with  RPP  to  merge  the  mission  areas.  If  this  is 
not  practicable,  the  DC/S  R&P  PEG  fills  this  role. 

(4)  The  DC/S  R&P  PEG  is  selected  by  RPP.  It  provides 
judgments  for  merging  mission  lists  and  mission- 
cluster  lists  from  sponsors. 

(5)  All  prioritization  activities  are  coordinated  by 
DC/S  R&P  (RPP) . 


3.3.2  Prioritization  authority  - 

(1)  The  PWG  and  PEG  have  the  final  authority  to  set 
program  priorities  during  the  prioritization  stage 
of  POM  development.  This  is  because  the  prioriti¬ 
zation  stage  merely  sets  the  start  point  for  the 
construction  stage.  It  is  during  the  construction 
that  initiatives  receive  their  final  place  in  the 
progreim,  and  then  the  Chief  of  Staff's  Committee 
is  the  final  arbiter  of  initiatives'  priorities  in 
the  POM  subject  to  the  Commandant's  approval. 

(2)  During  the  prioritization  stage,  the  PEG  determines 
the  relative  priorities  of  the  initiatives  it  samples 
from  the  mission  areas  and  m.ission  clusters.  That 

is  its  sole  function.  To  insure  that  its  judgments 
are  not  arbitrary,  its  work  must  include  a  system 
for  full  communication  and  feedback  with  the 
sponsors . 

(3)  The  sponsor-designated  experts  for  a  mission  area 
have  authority  to  establish  the  priorities  and 
mission  benefits  of  initiatives  in  their  mission 
area,  but  these  are  subject  to  adjustment  when  the 
PEG  examines  sample  initiatives  from  the  area. 

All  initiatives,  not  just  the  samples,  may  have 
their  mission  benefits  adjusted  to  be  consistent 
with  the  PEG'S  evaluation  of  the  samples.  Such 
adjustments  are  ir.ade  by  RPP;  they  may  be  appealed 
to  the  PWG. 

(4)  Sponsors'  expert  representatives  have  authority  to 
determine  how  the  mission  areas  represented  by  the 
sponsor  are  merged,  but  the  combined  priority  list 
is  subject  to  adjustments  when  sample  initiatives 
are  prioritized  by  the  PEG.  RPP  makes  such  adjust¬ 
ments  . 
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4 . 0  APPLICATION  PROCESS 


♦ 


This  section  describes  the  working-level  application  of 
decision  analysis  to  MC  POM  development.  Those  MC  officers 
or  contractor  personnel  who  apply  it  are  called  "decision 
analysts,"  and  their  function  is  called  "decision-analytic 
support."  The  section  describes  in  order; 

1.  the  decision  analysts*  and  MC  staff's  roles, 
general  tasks,  and  mode  of  work  for  decision- 
analytic  support; 

2.  the  MC ' s  technical  approach  for  eliciting,  quanti¬ 
fying,  and  merging  officers’  judgments  of  program, 
priori  ties ; 

3.  the  procedures  the  decision  analysts  follow  when 
working  with  mission  experts  and  the  PEGs  to 
quantify  and  combine  program  priorities;  and 

4.  the  MC ' s  technical  approach  and  procedures  for 
cost-benefit  analysis. 


Roles  for  Decision-Analytic  Sui 


The  decision  analysts,  MC  sponsors,  PEG,  and  DC/S  R&P 
RPP  assume  distinct  roles  in  the  application  of  decision 
analysis.  These  are  illustrated  schematically  in  Figure  4-1 


4.1.1  MC  sponsors  - 


(1)  The  MC  sponsors  provide  representatives  who 
act  as  experts  on  the  individual  mission  areas.  In  meetings 
conducted  by  RPP  and  the  decision  anlaysts,  these  represents 
tives  provide  judgments  about  the  priorities  of  program 
initiatives. 
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Provide 

•Benefits 

•Rationale 

•Costs 

•Briefs  on  selected  programs 
to  the  PEGs 


Provides 

•Benefits 

•Rationale 

•Mission  area  measures 
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•Assistance  for  problem 
definition 

•Benefit  assessments  from 
sponsors  and  PEGs 
•Analysis  and  prioritization 
of  programs 
•Mission  area  analysis 
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Figure  4-1 

OVERALL  ORGANIZATION 
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(2)  The  sponsors  may  also  provide  PEGs  to  assist 
the  decision  analysts  and  RPP  to  merge  mission  areas  by 
cluster. 


(3)  The  sponsors  provide  briefings  to  PEGs  on 
the  items  sampled  from  their  missions  or  cluster  of  missions. 

4.1.2  PEGS  -  An  official  DC/S  R&P  PEG  provides  cross¬ 

mission  expertise  for  merging  the  sponsors'  prioritized  sets 
of  initiatives.  This  PEG  may  also  be  employed  to  merge  a 
sponsor's  mission  areas,  at  the  sponsor's  request.  Alterna¬ 
tively,  the  sponsor  will  convene  an  ad  hoc  PEG  to  merge 
missions.  Thus,  there  may  be  several  PEGs.  The  functions  of 
all  PEGS  are:  (1)  to  hear  briefings  on  selected  initiatives 

that  represent  different  itiissions;  (2)  to  provide  judgments 
of  priority  among  the  selected  initiatives;  (3)  to  consider 
objections  from,  the  sponsors  about  these  priorities;  and 

(4)  to  assign  final  priorities. 

4.1.3  RPP  -  The  RPP  branch  coordinates  all  decision- 
analytic  support,  provides  briefings  to  sponsors  and  PKG  on 
the  support  methodology,  operates  the  POM-DBMS,  and  prepares 
intermediate  and  final  decision-analytic  support  reports  for 
the  PWG  and  sponsors.  Also,  the  RPP  branch  chairs  all 
meetings  of  the  PEGs  when  they  are  setting  program  priorities; 
in  addition,  the  RPP  chairs  all  meetings  of  sponsors'  repre¬ 
sentatives  when  they  are  setting  program  priorities  within 
individual  missions. 

4.1.4  Decision  analysts  -  The  decision  analysts'  role 
is  to  carry  out  the  MC’s  technical  approach  to  prioritization 
(Section  4.3)  and  also  to  continue  to  improve  the  approach. 

The  analysts  meet  with  RPP  and  the  sponsors'  representatives 
to  quantify  prograiri  priorities.  They  meet  with  RPP  and  the 
PEGs  to  merge  mission  priorities  quantitatively.  The  analysts 
design  functional  imiprovements  to  the  POM-DBMS  when  needed  by 
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RPP,  and  they  manage  the  system's  maintenance.  They  provide 
methodological  consultation  to  RPP  for  the  continued  develop 
ment  of  the  POM  methodology.  The  analysts  provide  methodolo 
gical  briefings  as  needed,  as  well  as  summary  reports  and 
appraisals  of  the  process. 

4 . 2  General  Tasks  and  Mode  of  Decision-Analytic  Support 

4.2.1  Tasks  -  Prioritization  is  the  major  task  of 
decision-analytic  support;  two  secondary  tasks  are  mission- 
area  analysis  and  software  development.  The  prioritization 
task  (described  in  Section  4.3)  occurs  several  times  as  RPP 
develops  the  military  manpower,  civilian  manpower,  O&M,  and 
procurement  programs;  it  may  also  occur  when  the  DC/S  Avia¬ 
tion  develops  the  blue-dollar  aviation  program.  Figure  2-2 
shows  the  schedule  of  green-dollar  prioritization  for  POM  83 

Mission-area  analysis  (MAA)  is  applied  after  the 
green-dollar  POM  is  approved  by  the  Commandant.  It  is  a 
retrospective,  quantitative  analysis  designed  to  show  the 
mission-area  emphases  of  the  POM  submission  at  the  minimum, 
basic,  enhanced,  and  over-guidance  levels.  It  identifies 
the  deficiencies  that  remain  in  each  mission  at  each  level. 
To  date,  MAA  has  been  used  only  experimentally  by  RPP  to 
learn  what  insights  it  might  provide  to  improve  planning  for 
the  subsequent  POM.  It  may  or  may  not  be  continued  in  its 
present  form;  consequently,  it  is  not  described  here.  A 
description  of  the  current  approach  is  provided  in  Waslov 
and  Kuskey  (1980),  cited  in  Section  2.4. 


Software  development  for  the  POM-DBMS  must  keep 
pace  with  changes  in  the  technical  approach,  working  proce¬ 
dures,  and  data  requirements  for  POM  preparation.  Part  of 
decision-analytic  support  is  to  develop  functional  specifi¬ 
cations  for  changes  to  the  system  and  to  supervise  the 
consequent  software  maintenance. 


4.2.2  Mode  of  support  -  Apart  from  software  development, 
the  great  majority  of  the  decision  analysts'  work  consists  of 
person-to-person  interactions  with  the  MC  staff.  Very  little 
independent  analysis  is  done  by  the  analysts.  In  particular, 
programmatic  recommendations  are  never  made  by  the  analysts. 
Their  recommendations  are  limited  to  the  methodology,  not  the 
content  of  programming. 

The  decision  analysts  work  with  up  to  thirty 
different  groups  of  MC  officers.  They  have  one  to  three 
meetings  with  each  group;  the  meetings  last  one  to  four  hours. 
Several  two-day  conferences  are  led  by  the  analysts  to  work 
with  the  PEGS.  Because  so  many  groups  participate  in  POM 
prioritization,  the  decision  analysts  must  essentially  be  on 
call  to  RPP  to  be  scheduled  at  RPP's  convenience. 

4 . 3  Methods  and  Procedures  for  Prioritization 

4.3.1  Introduction  -  We  distinguish  four  phases  of 
prioritization  that  require  different  methods  and  procedures 
of  decision-analytic  support: 

1.  program  packaging; 

2.  rank  ordering  and  scaling; 

3.  merging  scales;  and 

4.  cost-benefit  analysis. 

In  concept,  these  phases  follow  one  another.  In  practice, 
there  is  considerable  feedback  and  iteration  eunong  all  the 
phases;  moreover,  there  is  much  parallel  progress. 

While  program  packaging  is  often  the  critical 
first  step  in  a  successful  prioritization,  methodology  to 
support  it  can  best  be  explained  within  the  context  of 
scaling.  Hence,  it  is  discussed  as  a  special  topic  in  scaling. 
Scaling  and  merging  are  discussed  first  in  terms  of  theory  and 


then  again  in  practical  terms.  The  cost-benefit  analysis  is 
discussed  separately  at  the  end  of  the  section. 

4.3.2  The  technical  approach  to  preference  assessment  - 
Let  X  be  the  set  of  all  items  considered  in  the  MC  POM.  Let 
denote  the  items  that  are  proposed  for  mission  area  i; 
let  Xj^j  be  the  item  in  subset  X^^.  (For  the  Marine 

Corps,  each  item  appears  in  only  one  mission  area.  Thus,  X 
is  partitioned  by  the  subsets  Xj^.) 

The  Marine  Corps'  problem  is  to  develop  a  priority 
order  for  all  items,  i.e.,  a  complete  order  for  X.  They 
approach  this  problem  by  first  developing  a  complete  order 
for  each  subset  X^  and  then  merging  the  ordered  subsets. 

To  merge  the  subsets  efficiently  and  effectively,  a  quantita¬ 
tive  representation  (scale)  of  preferences  is  used.  The 
analyst  derives  this  representation  from  the  Marines'  non- 
numerical  answers  to  questions  about  items.  These  questions 
ask  for  judgments  of  the  importance  of  various  combinations 
of  items  relative  to  other  combinations.  The  required  re¬ 
sponses  are  "less  important,"  "more  important,"  or  "equally 
important."  The  experts  are  asked  to  compare  the  utility  of 
pairs  of  programs  and  relate  judgments  such  as:  A  is 

preferred  to  B;  B  has  more  utility  than  C  and  D  com¬ 
bined;  four  programs  like  E  are  approximately  equivalent 
to  F . 


The  analyst  can  construct  a  representation  of 
preferences  among  items  that  agrees  with  the  judgments.  The 
analyst  develops  quantitative  scales  for  each  mission  area 
and  for  sample  items  across  mission  areas.  The  sample  items 
are  used  to  establish  weights  for  the  mission  area  scales 
that  will  put  them  all  on  the  same  overall  Marine  Corps 
scale. 


The  method  of  scaling  is  based  upon  a  mathemati¬ 
cal  model  of  the  expert's  preferences  and  assumes  a  numerical 


scale  of  measurement  for  the  utility  of  the  items  in  the 
program  set.  The  scale  values  represent  the  relative  benefit 
or  utilities  of  the  programs  so  that  "Program  A's  value  is 
100"  does  not  mean  that  A  has  100  "utiles"  of  absolute 
value.  The  statement  means  that  it  would  take  100  programs 
with  a  value  of  1  to  obtain  the  same  utility  as  Program  A, 
or  two  programs  with  a  value  of  50,  etc.  The  scale  is 
constructed  so  that  multiplication  of  each  scale  value  by  a 
positive  constant  preserves  the  preference  order.  This 
capability  is  a  necessary  characteristic  of  the  model  because 
of  the  mission  area  structure  of  the  MC  programs.  Each 
mission  area  constructs  its  own  scale  of  relative  benefits, 
and  the  scales  are  merged  by  cross-multipliers.  A  multi¬ 
plier  is  a  numerical  representation  of  the  utility  of  the 
programs  in  one  mission  area  relative  to  those  in  the  other 
mission  areas. 

The  prioritization  methodology  is  a  useful  tool 
for  situations  in  which  several  sets  of  similar  programs  are 
competing  for  a  pool  of  resources  and  a  priority  order  across 
all  programs  needs  to  be  established.  It  utilizes  the  judg¬ 
ments  of  specialists  about  the  value  of  programs  within  their 
own  area  of  expertise  and  permits  a  comiriittee  of  generalists 
to  m.erge  all  the  program,  sets  in  a  simple  and  efficient  manner. 
The  approach  divides  the  task  of  value  assessment,  thereby 
reducing  the  cognitive  strain  that  can  occur  when  an  individual 
must  evaluate  and  compare  many  items  simultaneously. 

Benefit  assessment  is  also  useful  for  evaluating 
the  relative  cost-effectiveness  of  items  or  sets  of  items. 

When  given  a  specific  fiscal  constraint,  one  can  select  the 
most  beneficial  set  of  programs  to  fund  by  computing  cost- 
benefit  ratios  or  net  benefits  (see  Section  4.3.4). 

The  remainder  of  this  section  first  describes 
the  scaling  and  merging  processes  in  terms  of  a  mathematical 
representation  model  and  explains  the  underlying  assumptions 
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of  the  model.  It  then  describes  the  general  procedures  for 
assessing  and  merging  preferences.  Finally,  it  presents 
special  techniques  for  program  packaging. 


4. 3. 2.1  Preference  representation  model  and 
general  procedure  -  To  establish  a  set  of  rules  for  assigning 
numbers  to  objects  in  X,  we  first  define  relations  among 
the  elements  of  X,  such  as  preference  orderings,  and  a 
representation  of  the  empirical  structure--X  and  its 
relations — by  a  numerical  structure,  such  as  the  set  of  real 
n\imbers  (R)  and  its  arithmetic  relations.  We  denote  the 
empirical  structure  of  X  and  its  preference  relations 
Ti  by  <x,  Ti,  T2,  ..>  and  the  numerical  structure  of 
R  and  its  numerical  relations  by  <^R,  S]^,  S2  .  . 

The  empirical  structure  can  be  represented 
by  a  numerical  structure  if  there  exists  a  correspondence  i 
that  maps  X  into  R  and  maps  into  for  all  i. 

The  model  which  represents  the  MC  priori¬ 
ties  and  preferences  for  POM  programs  is  a  numerical  rela¬ 
tional  structure.  For  the  elements  x  and  y  of  X,  we 
define  tie  binary  relation,  "equal  or  more  important  than," 
denoted  by  ^  .  Then  x  X  y  means  x  has  equal  or  greater 
importance  than  y.  We  can  also  define  the  indifference 
relation  x  ~  y  to  mean  that  neither  x  nor  y  is  more 
important  than  the  other,  that  is,  x^  y  and  y^  x. 

In  addition  to  stating  the  preference 
order  of  individual  programs  x^  yb  Zr  the  MC  states  pre¬ 
ferences  for  combinations  of  programs  such  as  (y  and  zb  x) 
or  (w  and  y  b  x  and  z) .  A  combination  of  the  elements  of 
X  is  the  simultaneous  purchase  of  two  or  more  programs  and 
is  denoted  by  0.  The  combination  operation  is  represented 
by  the  operation  of  addition  in  the  model;  and  the  binary 
relation  "equal  or  more  important"  (b)  is  represented  by 


r 
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the  arithmetic  comparison  "greater  or  equal"  (>) .  That  is, 

■<Cx,  b  /  is  represented  by  <[r, 

Two  topics  are  to  be  addressed  concerning 
this  representation:  the  logical  properties  of  the  numerical 
assignment  necessary  to  make  it  suitable  for  prioritization, 
and  the  procedures  for  making  the  numerical  assignments. 

Properties  -  We  seek  assumptions  concerning 
t  and  c  that  are  sufficient  to  construct  a  function,  (} , 
that:  (1)  maps  all  elements  of  X  and  all  possible  combinations 
of  the  elements  of  X  into  R;  and  (2)  satisfies: 

(i)  y  if  and  only  if  C  (x)  ^  i  (y) 

and 

(ii)  d  (x  c  y)  =  d  (x)  +  c  (y)  . 

We  also  seek  the  admissible  transformations  ^  C '  that 
yield  homomorphisrrs  into  <^R,  +>.  If  d  satisfies  these 

relationships,  it  will  be  useful  not  only  for  comparing  one 
POM  initiative  or  current  capability  to  another,  but  also 
for  comparing  any  set  of  initiatives  to  any  other  set. 

The  set  of  axioms  sufficient  for  the 
representation  are  given  in  Krantz,  et  al.  (1971)  as  well  as 
the  proof  of  sufficiency.  For  the  structure  <Cx,i’  ,  ^  with  X 
nonempty,  L  ,  a  binary  relation  and  o,  a  closed  binary 
operation,  they  prove  that  a  real-valued  function  ({>  on  X 
exists  which  satisfies  (i)  and  (ii)  if  four  axioms  hold 
for  all  X,  y,  z,  w  t  X,  that  is,  for  all  elements  in  the 
set  X.  The  four  axioms  are: 


1. 


is  a  connected  and  transitive  relation. 


2.  X  0  (y  0  z)  ^  (x  o  y)  0  z, 

3.  xb  y  if  and  only  if  {x  o  z  ^  y  o  z)  and 

(z  0  X  t  z  c  y)  , 

4.  If  xlH  y,  then  there  exists  a  positive  integer  n 
such  that  n  y's  are  at  least  as  important  as  x. 

» 

The  first  axiom  specifies  that  the  empiri¬ 
cal  structure  is  a  weak  order:  connectedness  m.eans 

that  for  all  x,  y,  z  c  X,  either  x  y  or  y  b  x.  Trans¬ 
itivity  means  that  if  xh  y  and  y  z,  then  x  i  z. 

These  conditions  are  rarely  violated  once  programs  are  properly 
packaged.  They  guarantee  that  c  is  an  ordinal  scale  (it 
preserves  the  rank  order  of  initiatives)  when  X  is  a  finite 
set. 


The  second  axiom  requires  that  the  relation 
t  be  associative.  Since  we  define  combination  simply  as 
the  decision  to  purchase  all  the  combined  items,  the  order 
of  combination  can  be  arbitrary. 

The  third  axiom  is  the  monotonicity  property 
and  is  also  called  the  cancellation  law.  It  requires  that 
the  elements  of  X  be  independent  in  value  so  that  the 
purchase  of  an  item  does  not  cause  a  change  in  preference 
order  by  increasing  or  decreasing  the  value  of  other  items. 

For  the  POM  program  structure,  this  axiom  can  be  violated  in 
two  ways  that  require  special  treatment: 

(1)  The  purchase  of  items  may  have  a  synergistic  effect. 
For  example,  one  might  prefer  item  a  over  item  b, 
but  then  prefer  the  combination  of  items  b  and  c 
over  items  a  and  c  because  b  and  c  interact 
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to  produce  added  benefit;  then  a  ~  b  does  not 
imply  (a  0  c)  (b  c  c)  and  axiom  3  does  not  hold. 

(2)  Items  are  compensatory  so  that  the  purchase  of  one 
neutralizes  or  decreases  the  value  of  the  other. 

This  effect  can  occur  when  similar  programs  are 
substituted  for  one  another  and  the  purchase  of 
both  is  duplicative.  For  example,  a  is  preferred 
to  b,  or  a  h  b,  but  a  and  c  are  compensatory 
so  that  (bo  c) b  (a  c  c) ,  violating  axiom  3. 

The  Archimedean  axiom,  axiom  4,  requires 
that  any  two  elements  of  X  must  be  comparable,  i.e.,  the 
importance  of  an  item,  must  not  be  infinitesimally  small  (or 
large)  relative  to  any  other  item,.  For  instance,  a  set  of 
"must  buy"  program, s  that  cannot  be  traded  off  for  other  pro- 
gramis  cannot  be  represented  by  the  model. 

The  tests  for  validation  of  these  four 
axioms  occur  during  the  construction  of  the  scale  which  is 
described  in  Section  4. 3. 2. 2. 

It  follows  directly  from  the  representation 
theorem  that  for  any  o  >  0,  the  function  (i’  =  af  also 
satisfies  the  axioms.  Thus  multiplication  of  the  scale  f  by 
a  constant  produces  another  scale,  <P ' ,  which  provides  an 
equivalent  representation  of  the  structure 

The  multiplicative  transformation  is 
useful  in  the  construction  of  the  benefit  scale  for  the  POK 
progreun  set.  We  first  construct  individual  scales, 
i  a  1,  2,  ...,  n  for  each  of  the  n  mission  areas.  Then 
we  seek  positive  multipliers,  k^,  for  which  an  overall 
scale.  If,  can  be  defined  as  (f  =  k^^ 
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To  obtain  the  kj^,  we  select  a  set,  X®, 
of  sample  items,  one  item,  Xj^  ,  from  each  mission  area.  If 
the  four  axioms  hold  for  the  structure  ^  we  can 

construct  a  function  C  that  maps  X^  into  R.  The  scale 
^  describes  the  empirical  relationship  of  items  across  all 
mission  areas,  and  allows  us  to  derive  by: 


C(x.“) 


♦i(Xij) 


;(xj) 

Thus,  k.  = 


The  properties  of  $  are  that:  (1)  i  (x^^)  =  k^(J (x ;  (x^ 
and  (2)  is  equivalent  to  Cj^  for  any  given  mission  area. 
Because  of  these  properties,  it  is  assumed  that  all  initia¬ 
tives  are  properly  scaled  by  i  to  the  extent  that  all  the 
(f  i  and  z,  functions  are  developed  well,  though  most  of  the 
initiatives  have  never  been  directly  compared  across  mission 
areas . 


The  procedures  for  the  construction  of 
the  scales,  and  $,  and  the  assignment  of  numerical 

values  to  items  are  described  below.  A  more  detailed  descrip¬ 
tion  is  given  in  Section  4. 3. 2. 2  with  an  illustration  of 
benefit  assessment  tactics. 

General  Procedures  -  The  procedure  for 
defining  ,  the  overall  benefit  scale,  begins  with  the 
construction  of  individual  m.ission-area  scales,  4)^^  . 

Decision  analysts  meet  with  mission-area  representatives  and 
elicit  their  evaluations  of  the  programs  in  their  mission 
area.  The  representatives  are  given  brief  instructions; 
then  they  compare  various  combinations  of  the  programs  using 
the  following  criteria: 
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o 


well-defined  progran\(s)  ; 
o  clear  requirement  and  concept  of  employment; 

o  military  effectiveness; 

o  breadth  of  application; 

o  current  versus  new  capability; 

o  readiness  versus  modernization; 

o  proper  program  scheduling; 

o  technical  risk. 

The  necessary  comparative  judgments  are  simply  "yes"  or  "no, 
answers  that  determine  whether  or  not  a  set  of  one  or  more 
items  has  equal  or  greater  value  than  another  set. 

The  first  task  is  to  establish  a  rank- 
order  of  individual  items  ir.  Xj^  so  that 


with  the  superscript  denoting  the  rank  of  the  item  in 
Then  the  analysts  present  a  series  of  questions  concerning 
preferences  for  combinations  of  items  in  order  to  narrow  the 
interval  (on  the  scale)  which  contains  a  given  item. 

For  excuuple,  we  might  find  that  o  V  x^^  ^  (x^^  c  , 

which  implies  that 

+  (f^(x.^)>  (4-1) 

If  we  let  ~  100  be  an  arbitrary 

reference  point,  then  (tj^(Xi^)  =  75,  <f^(xi3)  =  50,  and 

({!^(xi^)  =  25  is  one  set  of  numbers  that  satisfies  equation 
(4-1) .  Theoretically,  if  the  axioms  held  and  enough  inequali¬ 
ties  were  stated,  they  could  be  solved  for  a  unique  4)i. 

In  practice,  there  is  no  need  to  elicit  all  the  judgments 
necessary  for  a  unique  representation.  An  approximate 
representation  is  adequate.  So,  the  group  derives  a  few 
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inequalities  and  assigns  numbers  that  satisfy  the  inequali¬ 
ties.  Ihe  implications  of  any  given  assignment  can  be 
tested  by  assessing  preferences  for  other  combinations  of 
items,  and  the  numbers  can  be  refined  as  the  elicitation 
process  evolves.  (Techniques  for  questioning  and  assigning 
numbers  are  described  in  Section  4. 3. 2. 2.) 

After  the  individual  m.ission-area  scales, 
are  constructed,  they  must  be  merged  by  a  cross-scale. 
It  would  be  possible  to  select  one  item  from  each  m.ission 
area  and  produce  one  cross-scale  ;  by  eliciting  the  PEG'S 
preferences  for  these  items,  but  for  practical  reasons,  the 
merging  is  done  hierarchically.  The  mission  areas  are 
clustered  by  sponsor,  ai'.d  each  sponsor  produces  a  cross¬ 
scale  that  represents  preferences  for  items  from  the  mission 
areas  within  its  cluster.  All  of  the  sponsors'  cross-scales 
are  subsequently  merged  by  another  cross-scale  produced  by 
the  Program,  Evaluation  Group.  See  Figure  2-1  for  a  display 
of  the  clusters. 

The  process  for  producing  a  cross-scale 
is  the  same  at  both  the  sponsor  level  and  the  PEG  level  of 
the  hierarchy.  First,  we  form  a  set  Y  by  selecting  three 
items--one  high-,  one  medium-,  and  one  low-priority  item-- 
from  each  cluster  to  present  to  the  group  of  evaluators 
(PEG) .  This  set,  Y,  is  partitioned  into  a  high-priority 
item  subset,  Yjj;  a  miedium-priority  item  subset,  and 

a  low-prior \ty  item  subset,  Y^^.  Each  subset  is  scaled 
separately  by  the  PEG,  and  then  the  three  scales  are  merged 
to  produce  one  overall  scale,  ;  . 

The  three  separate  evaluations  of  items 
from  one  cluster  or  mission  area  allow  the  PEG's  judgments 
about  the  ratios  of  value  aunong  the  three  items  to  be  com¬ 
pared  to  the  original  judgments  of  the  mission  area  commit¬ 
tees  (MACS)  .  Formally,  denote  the  three  items  chosen  fromi 


cluster  i  as  X  ,  and  x^^  .  The  comparison  of  PEG 

and  MAC  judgments  can  be  made  by  checking  the  validity  of 
the  following; 

Ux.^) 

■  ■  S  ■  IB  ■  — 

$i(Xi^)  4>i(x^^) 


where  the  numerators  are  the  PEG  scale  values  for  the  three 
subsets  representing  three  priority  levels.  When  the  equality 
does  not  hold  for  items  of  a  cluster  or  mission  area,  then  the 
members  of  the  respective  committees  convene  to  resolve  the 
conflict  and  revise  the  scales  to  be  compatible. 

Figure  4-2  charts  the  progression  of  the 
scaling  tasks:  starting  with  a  set  of  MC  programs,  X, 

(1)  Mission  Area  Comir.ittees  construct  scales,  <t 

(2)  Cluster  PEG'S  construct  scales,  ;,  by  merging 
Missions  into  clusters; 

(3)  MACS  and  cluster  PEGs  adjust  their  scales; 

(4)  PEG  constructs  scale,  Cf  for  sample  items  from 
the  clusters; 

(5)  MACS,  cluster  PEGs,  and  the  PEG  adjust  their 
scales;  and 

(6)  a  scale,  g,  is  derived  which  reflects  a  priority 
order  for  all  programs  in  X. 

4 . 3 . 2 . 2  Single-scale  assessment  procedures  -  This 
section  describes  the  procedures  used  by  decision  analysts  to 
derive  the  benefit  scales  and  for  each  mission  area. 

We  discuss  several  different  approaches  through  the  use  of  the 
following  example. 


Figure  4-2 

THE  SCALING  PROCESS 
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Suppose  an  analyst  is  assisting  an  MC  com¬ 
mittee  with  the  scaling  of  nine  items  X  =  A,  B,  C,  I. 

The  group  has  already  ranked  the  items  in  order  of  their  value 
with  respect  to  relevant  criteria,  and  C~  I. 

The  analyst's  task  is  to  elicit  the  judgments  of  the  group 
about  the  value  of  the  items  in  such  a  way  that  a  scale 
can  be  derived  by  the  solution  of  a  set  of  inequalities  from, 
the  rules: 

(i)  If  X  ^  y,  then  $ (x)  ^ $ (y) ,  and 
(ii)  (x  c  y)  =  (J  (x)  +  C  (y)  .  (4-2) 

Top-down  comparison  questions  -  The  pri¬ 
mary  method  for  constructing  a  benefit  scale  is  by  top-down 
comparisons  of  items  and  bottom-up  numerical  assignments. 

This  approach  is  motivated  by  the  belief  that  judgments  about 
the  comparative  value  of  items  are  most  easily  made  for  items 
that  are  close  in  value.  So,  it  is  easier  to  assess  the 
value  of  iterri  B  relative  to  item  A,  than  relative  to  lower- 
priority  items  like  G,  H,  and  I. 

The  analyst  presents  the  group  with  choices 
between  individual  items  and  combinations  of  items,  beginning 
at  the  top  of  the  rank-order  with  item  A.  Item  A  is  compared 
to  the  combination  of  items  B  and  C,  and  the  group  must 
decide  whether  they  would  prefer  the  purchase  of  A  alone,  or 
the  purchase  of  both  B  and  C.  Suppose  the  group  decides 
A^  (B  c  c)  .  The  analyst  then  makes  the  selection  of  A  less 
attractive  by  offering  B  and  C  and  D  together  versus  A. 

Questions  about  A  continue  until  an  indifference  point  is 
reached,  e.g.,  A~(B  o  c  o  d)  ,  or  the  preference  reverses, 
e.g.,  (B  o  c  0  D) h  A.  Suppose,  on  the  other  hand,  that 
(B  c  C)  i  A.  Then,  the  analyst  makes  the  preferred  side  less 

attractive  by  offering  B  and  D  versus  A,  or  C  and  D 

versus  A.  Again,  the  comparisons  continue  until  an  indif¬ 
ference  point  or  a  reversal  is  found.  An  interval  about 
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0(A)  will  be  established  by  the  implications  of  the  repre¬ 
sentation  model.  For  example,  if  (C  o  D  o  E)  ^  A  ^  (C  c  D) , 
then 

0(C)  +  0(D)  +  0(E)  ^  0(A)  ^  0(C)  +  0(D). 

In  this  process,  note  that  it  is  the  point  of  reversal  or 
equality  that  establishes  upper  and  lower  bounds  for  0 (A) 
in  relation  to  suir.s  of  0(C),  0(D),  etc.  Once  0(A)  is  so 
bracketed,  the  next  step  is  to  establish  an  interval  about 
0(B),  then  0(C),  and  so  forth. 

Numerical  assignment  -  The  ease  of  the 
subsequent  numerical  assignment  will  depend  on  the  number  of 
preference  relations  elicited  and  the  tightness  of  the  inter¬ 
val  about  each  item.  With  respect  to  the  latter  criterion, 
the  analyst  should  strive  to  bound  0 (x)  as  narrowly  as 
possible  to  reduce  the  size  of  the  solution  space.  For 
example,  the  preference  relation  (D  c  E  o  F)  ^  B  ^  (C  c  D) 
provides  more  information  about  0 (B)  than  does  the  relation 
A  B  >  (EC  F)  . 

Suppose  that  in  the  example,  the  analyst 
elicited  the  following  preference  relations: 


(1) 

(B  0 

C)  ^  A 

(2) 

A  > 

(C  0  D) 

(3) 

B  ~ 

(C  C  D) 

(4) 

c  >- 

(D  G  E) 

(5) 

(D  0 

E  0  F)  >“  C 

(6) 

D  ~ 

(E  0  F) 

(7) 

(E  0 

F  0  G)  >-  D 

(8) 

E  ~ 

(F  0  G) 

(9) 

(H  0 

I)  >-  F 

(10) 

(H  0 

I)  >-  G 
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These  relations  are  represented  by  inequalities  in  the  model 
and  can  be  solved  for  4)  (A)  ,  41  (B)  ,  4(1)-  In  order  to 

derive  4,  we  first  fix  the  value  4(1)  arbitrarily  at  10. 
Then  we  determine  the  ratio  of  value  between  H  and  I  by 
asking,  "How  many  initiatives  like  I  would  it  take  to 
provide  the  same  importance  as  H?"  Suppose  the  response  is 
2,  so  that  do  I)  ~  H,  and  2  4  (I)  =  4  (H)  , 

At  this  point,  the  inequalities  implied 
by  the  relations  can  be  solved,  but  not  uniquely,  for  a  set 
of  scores.  The  equations  and  inequalities  are: 

(1)  e  (A)  <  4  (B)  +  4  (C) 

(2)  4  (A)  >4  (C)  +  :  (D) 

(3)  4  (B)  =4  (C)  +  4  (D; 

(4)  4  (C)  >  4  (or  +  4  (E) 

(5)  4  (C)  <  4  (D)  +  4  (E)  +4  (F) 

(6)  4  (D)  >  4  (E)  +  4  (F) 

(7)  4  (D)  <  4  (E)  +  4  (F)  +4  (G) 

(8)  4  (E)  +4  (F)  +  4  (G) 

(9)  4  (F)  <  4  (H)  +  4  (I) 

(10)  4  (G)  <4  (H)  +  4  (I) 

(11)  4  (K)  =  20 

(12)  4  (I)  =  10 

The  analyst  still  has  a  great  deal  of 
flexibility  in  assigning  numbers  that  solve  the  inequalities, 
and  one  might  reduce  the  solution  space  by  presenting  further 
comparison  questions.  Typically,  however,  one  discovers  the 
best  comparison  questions  to  ask  by  going  directly  on  to  the 
assignment  of  numerical  values.  The  analyst  relates  the 
items'  intervals  of  value  that  are  implied  by  the  preference 
relations,  and  the  participants  are  asked  to  select  the  bene¬ 
fit  values.  In  the  example,  we  know  that  20  <  4(G)  <  30, 
and  the  group  decides  that  4(G)  =  25.  Now,  25  <  4(F)  <  30, 
and  the  group  decides  that  4(F)  =  26.  4(E)  is  calculated 


to  be  51,  so  77<(Ji(D)  <  102.  Here,  the  group  has  a  wider 
range  of  values  from  which  to  choose,  and  decides  that  D  is 
closer  in  value  to  E  o  F  o  G  and  assigns  it  a  benefit  of 
100.  Subsequently,  they  let  <1)(C)  =  160,  ${B)  =  260,  and 
(A)  =  300.  The  resultant  scale  is: 


Item 

A 

B 

C 

D 

E 

F 

G 

H 

I 


(js  (Item) 
300 
260 
160 
100 
51 
2  6 
25 
20 
10 


The  analyst  uses  the  numerical  scale  to  suggest  new  compari¬ 
sons.  For  instance,  two  comparisons  suggested  by  the  above 
numbers  are: 

(1)  (Be  D)  A,  and 

(2)  D  h  (F  c  G  0  H)  ,  etc. 

Direct  adjustments  to  the  values  are  made  as  a  result  of 
these  questions,  always  making  sure  that  i.  satisfies  the 
earlier  judgments  as  well. 

An  approach  fo r  seal ing  long  1 i s t s_o f 
items  -  A  variation  to  the  top-down  approach  to  scaling  is 
often  used  when  the  number  of  items  to  be  scaled  exceeds  1.'. 
In  top-down  questioning,  the  analyst  presents  a  series  of 
paired  comparisons  until  an  indifference  or  reversal  point  is 
found;  whereas  in  the  variation,  the  analyst  directly  asks 
the  group  to  find  the  reversal  point.  Given  that  A  is  the 
first  priority  in  a  rank-ordered  list  of  20  items  A  through 
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T,  the  group  is  asked  to  exeiinine  the  list  in  descending  order 
and  find  the  first  two  consecutive  items  x  and  y  that  give 
A  ^  (x  0  y) .  If  the  selected  items  were  I  and  J,  the 
group  is  then  asked  to  find  the  first  two  consecutive  items 
w  and  z  that  give  I  h  (w  c  z) .  This  process  continues 
until  no  more  of  these  comparisons  can  be  made. 

The  approach  provides  a  simple  way  to 
start  the  scaling  of  a  long  list  of  items;  it  saves  time  by 
requiring  fewer  comparison  questions.  The  result  is  a  list 
partitioned  by  break-points  that  can  be  used  to  divide  the 
items  into  smaller  lists  of  about  10-15  items.  Each  list  is 
scaled  separately  by  top-down  comparisons  and  bottom-up 
assignments,  then  quickly  merged  by  using  the  preference 
relation  that  determined  the  break-point. 

For  example,  suppose  the  list  of  20  items 
A  through  T  had  the  relations: 

1  t  3 
itho  K 
L  t  Q  c  R. 


The  analyst  can  divide  the  list  into  two — A  through  K  and 
L  through  T,  and  using  the  method  described  in  the  previous 
section,  separately  derive  the  following  two  scales: 


Item 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 


(Item) 

50 

48 

47 

45 

42 

40 

35 

30 

22 

18 

15 


Item 

L 

M 

N 

0 

P 

Q 

R 

S 

T 


<p  (Item) 
95 
90 
80 
62 
60 
50 
30 
25 
10 
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The  two  scales  can  be  merged  by  observing  that  I  h  L  o  M 
and  testing  a  few  other  combinations  such  as: 


L  0  M  ^  J 
J  ^  M  c  N. 


The  above  preference  relations  tell  us  that  we  can  merge  the 
lists  by  multiplying  the  first  list,  A  through  K,  by  the 
constant  a  =  10. 


A  faster  approach  for  experienced  partic¬ 
ipants  -  If  the  group  of  experts  has  had  previous  experience 
in  scaling,  the  process  can  proceed  more  quickly  by  allowing 
them  to  make  direct  numerical  assignments.  If  they  under¬ 
stand  the  properties  of  the  representation  model  (in  equation 
4-2) ,  they  can  place  benefit  values  on  the  items  directly 
without  the  aid  of  initial  comparison  questions. 

The  first  item  of  a  rank-ordered  list  is 
given  a  value  of  100,  and  the  other  items  are  scaled  relative 
to  it.  After  the  group  makes  the  assignments,  the  analyst 
reviews  several  preference  relations  that  are  implied  by  the 
scale  to  check  for  consistency.  The  process  of  numerical 
assignments  with  checks  by  the  analyst  iterates  until  the 
participants  judge  the  results  to  be  accurate. 

Special  cases  -  There  are  some  cases  for 
which  the  analyst  would  have  difficulty  using  the  described 
top-down  approach  for  elicitation  and  scaling,  because  the 
preference  relations  between  single  items  and  combinations 
do  not  give  sufficient  information  about  the  scale.  Two 
examples  are: 

1.  the  value  of  the  top-priority  item  exceeds  that 
of  all  other  items  combined  (this  is  essentially 
a  set-closure  problem) ;  and 
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2. 


all  the  items  of  the  list  are  so  close  in  value 
that  the  value  of  any  two  items  exceeds  the  value 
of  any  one  item;  thus  the  analyst  cannot  find  a 
lower  bound  for  each  4)(x). 

The  first  case  can  be  handled  in  several  ways.  The  analyst 
can  remove  the  excessively  high-valued  items  from  the  set  and 
scale  the  remainder  by  the  regular  method.  Then  he  can  try 
to  identify  the  ratio  of  value  between  the  high-valued  item 
and  the  highest  ranking  item  in  the  scaled  list,  and  get,  for 
exaiTiple,  A  ~  100  E.  This  type  of  ratio  judgm.ent  is  often 
difficult  to  elicit,  so  a  precise  numerical  assignment  is 
postponed  until  the  next  level  of  the  scaling  hierarchy 
(cluster  PEG  or  overall  PEG) ,  where  items  of  similarly  high 
value  from,  other  mission-area  sets  can  be  compared. 

For  the  second  case,  the  analyst  must 
present  comparison  questions  about  two  or  more  items  versus 
two  or  more  other  items  in  order  to  establish  enough  preference 
relations  for  numerical  assignment.  The  value  judgments  are 
more  difficult  to  make,  and  the  resultant  inequalities  are 
more  difficult  to  solve,  so  the  analyst  might  opt  to  request 
direct  numerical  assignments  and  then  check  the  results  by 
using  the  multiple  combination  comparison  questions. 

Consistency/validity  checks  -  Since  the 
benefit  scales,  will  undergo  later  revisions  that  are 

based  only  on  samples  from,  the  sets  Xj^,  it  is  very  important 
at  this  stage  for  the  analyst  to  ensure  that  the  preference 
relations  are  consistent  with  the  model's  axioms  and  that 
the  scale  accurately  reflects  the  elicited  judgments. 

Systematic  tests  for  axiom  violation  should 
always  be  made  during  the  elicitation  of  preference  relations. 
When  two  or  more  items  are  synergistic  or  compensatory,  the 
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group  will  have  difficulty  with  comparison  questions  that 
involve  these  items,  and  they  will  most  likely  cite  the 
existence  of  an  interaction  as  rationale  for  a  reluctance  to 
make  judgments.  In  such  a  case,  the  items  are  redefined  or 
their  elements  are  repackaged  so  that  items  that  have  inter¬ 
dependencies  are  grouped  together  as  one  program.  (Other 
approaches  to  program  packaging  are  described  later  in 
Section  4.3.3.) 


The  diagram  in  Figure  4-3  illustrates  the 
in-process  checks  for  consistency. 

4. 3. 2. 3  Multi-scale  merging  procedures  -  As  de¬ 
scribed  in  the  General  Procedures  part  of  Section  4. 3. 2.1, 
the  PEG  has  the  task  of  establishing  a  cross-scale  that 
merges  mission-area  or  cluster  scales.  The  merging  process 
begins  after  the  mission  areas  have  produced  scales  for 
their  items,  and  the  analysts  and  RPP  have  selected  high-, 
medium-,  and  low-priority  items  from  the  mission  areas  or 
clusters.  The  items  chosen  to  represent  a  mission  area 
frequently  include: 

1.  items  that  were  controversial  within  the  mission 
area; 

2.  items  that  had  large  benefit-values  that  could  not 
be  assessed  by  the  mission  area;  and 

3.  two  consecutive  items  from  a  mission  area  whose 
scale  is  characterized  by  two  or  three  closely 
scaled  groups  of  items  (e.g.,  a  scale  that  has 
five  items  with  benefit  values  between  90  and  100, 
and  five  items  with  benefit  values  between  10  and 
20)  . 


The  high-priority  items  are  usually  the  top  item  in  each 
mission  area.  The  other  items  are  chosen  somewhat  randomly 
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from  high,  medium,  and  low  segments  of  the  scale  with  a 
preference  for  items  that  are  well  defined.  These  items  are 
described  in  short  briefings  to  the  PEG  by  their  proponents. 

The  PEG  generally  produces  and  merges  three 
separate  scales,  one  for  the  high-priority  items  (called  the 
high  cut)  ,  one  for  the  medium-prio’-ity  items  (the  middle  cut)  , 
and  one  for  the  low-priority  items  (the  low  cut) .  These 
scales  are  usually  derived  by  the  top-down  questioning  approach; 
however,  the  difference  between  the  PEG  procedure  and  the 
mission-area  committee  procedure  is  that  the  PEG  devotes 
considerably  more  time  to  rank-ordering  and  numerical  assign¬ 
ment. 


Rank-ordering  -  Each  member  of  the  group 
independently  rank-orders  the  items  of  the  cut  under  con¬ 
sideration,  and  then  all  the  rankings  are  displayed.  The 
areas  of  greatest  conflict  are  discussed.  For  example,  if 
one  item  from  a  ten-item  list  receives  two  individual  ranks 
of  two  (second  priority) ,  and  eight  (eighth  priority) ,  then 
the  individuals  are  encouraged  to  explain  their  reasons  for 
the  disagreement  in  ranking.  Often,  the  cause  for  the 
conflict  is  a  lack  of  knowledge  of  certain  aspects  of  the 
programs.  In  such  cases,  more  discussion  about  the  items' 
characteristics  will  resolve  the  disputed  ranking.  After  the 
major  differences  in  rankings  are  discussed  and  information 
is  exchanged,  the  individuals  revise  their  rankings.  The 
cycles  of  ranking  and  discussing  differences  continues  until 
a  group  consensus  is  reached. 

Scaling  -  By  the  time  a  final  rank-order 
of  items  is  established,  the  group  will  have  a  good  under¬ 
standing  of  the  prograuns '  merits,  so  they  will  have  little 
difficulty  in  responding  to  the  analyst's  comparison  questions 
for  scaling.  The  analyst  can  use  the  typical  approach  of  top- 
down  questions  and  bottom-up  assignments,  or,  as  the  group 
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becomes  experienced  in  scaling,  direct  numerical  assignments. 
In  the  latter  case,  the  individuals  will  scale  the  ranked 
items  independently  and  discuss  the  results  and  their  differ¬ 
ences,  working  towards  a  consensus  in  the  same  manner  as  the 
ranking  procedure. 


Merging  the  three  cuts  -  After  the  three 
cuts  are  scaled,  they  are  merged  into  one  scale.  The  an  lyst 
asks  the  PEG  to  position  the  highest-priority  item  (or  any 
other  sample  item)  in  the  middle  cut  relative  to  the  items 
in  the  high  cut.  For  exam.ple,  the  sample  item  may  be  judged 
to  fall  between  the  seventh  and  eighth  priority  items  of  the 
high  cut,  and  thus  will  have  a  bene'^it  value  in  the  range 
between  their  values.  The  analyst  can  derive  the  specific 
value  fromi  a  few  com.parison  questions,  or  allow  the  group 
to  make  a  direct  nur.erical  assignment.  The  values  of  the 
other  items  in  the  middle  cut  are  then  multiplied  by  the 
ratio  of  the  sample  item's  score  in  the  high  cut  to  the 
sample  item's  score  in  the  middle  cut.  The  low  cut  items 
are  folded  into  the  scale  in  the  same  manner. 

Comparison  of  PEG  scale  and  mission-area 
scales  -  The  merging  of  the  three  cuts  gives  the  ratios  of 
value  among  the  three  sample  items,  x^^^,  from 

mission  area  i,  as  perceived  by  the  PEG.  The  PEGs  values 
are  compared  to  the  mission  area's  values  by  checking; 

C(x^®)  C(x^^) 

♦i(x^*^) 

The  equation  fails  to  hold  when  the  PEG  judges  the  items  to 
have  ratios  of  values  different  from  those  perceived  by  the 
mission  area.  For  example,  the  mission  area  may  have  assigned 
Xi^,  Xj^M,  and  x^^  the  values  90,  80,  and  20,  respectively, 
while  the  PEG  assigned  to  them  the  values  75,  30,  and  15. 
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Here,  there  is  little  difference  between  the  ratios  of  xi^ 

to  but  considerable  disagreement  about  the  ratio  of 

H  M 

Xf  to  •  (In  some  cases,  the  PEG  might  even  reverse  the 

items'  order  of  priority.) 

Disagreements  are  discussed  and  resolved 
during  subsequent  meetings  of  the  PEG  with  each  mission-area 
group.  The  groups  exchange  information  about  programs,  and 
the  analyst  asks  both  groups  to  revise  their  ratios  until  a 
consensus  is  reached.  (If  the  groups  cannot  agree,  the  PEGs 
scale  takes  precedence.) 

Revision  of  mission-area  scales  -  Once 
the  PEG  decides  on  the  final  values  c  (Xj^^)  ,  c  (Xj^^)  ,  and 
; (x^^)  for  each  mission  area,  the  analyst  revises  the 
mission-area  scales  to  reflect  the  final  values,  while  striving 
to  maintain  (approximately)  the  original  preference  relations. 

The  translation  of  the  original  mission- 
area  scale  to  a  revised  scale  is  shown  pictorially  in  Figure 
4-4.  The  values  of  the  sample  items  are  set  equal  to  the 
final  PEG  values  for  those  items.  The  revised  values  of  all 
other  items  in  the  mission  area  are  approximated  by  the  graph 
of  the  line  segm.ents  drawn  between  the  sample  points. 

Since  the  PEG  has  evaluated  the  benefit 
of  three  items  from  each  mission  area  relative  to  items  from 
each  of  the  other  m.issicn  areas,  the  cross-scale  is  established 
and  we  can  set  J  (the  final  scale  across  all  items  in  the 
POM)  to  equal  the  revised  mission-area  scale,  so  that  for  each 
mission  area,  i, 

(x,.  )  =  (x^)  ,  c  X^. 
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4. 3. 2. 4  Integrated  program  packaging  -  Special 
cases  exist  for  which  it  is  difficult  to  design  independent 
program  initiatives  that  meet  the  restrictions  of  the  model's 
axioms.  Examples  occur  in  the  MC  missions  of  tactical  com¬ 
munications,  command  and  control,  and  others.  The  axioms 
are  violated  due  to  interactions  between  programs;  thus,  the 
analyst  has  difficulty  deriving  benefit  scales.  Dependencies 
exist  because  these  missions  are  accomplished  through  systems 
and  networks  with  elements  like  links  in  a  chain.  It  is 
combinations  of  these  elements  that  provide  value,  not  the 
independent  elem.ents  themselves.  For  instance,  upgrading  a 
radio  transm.itter  alone  has  little  value  if  the  new  trans¬ 
mitter  will  not  function  with  the  existing  receivers.  The 
communication  system  must  have  an  integrated,  not  piecemeal, 
upgrade. 


Vvhen  independence  of  items  cannot  be 
assumed,  yet  a  program,  for  operating  or  acquiring  the  items 
must  be  developed,  then  the  item.s  are  packaged  together  into 
a  logical  set  of  system  initiatives.  The  first  initiative 
in  the  set  is  a  "bare  bones"  improvement  to  the  existing 
system.  The  next  initiative  is  a  fully  integrated,  system, 
enhancement  of  the  bare  bones  initiative.  The  third  initia¬ 
tive  is  an  enhancement  of  the  second,  and  so  forth.  Thus, 
one  defines  a  set  of  integrated,  executable,  staged  enhance¬ 
ments  of  the  status  quo  system,  extending  fromi  "bare  bones" 
to  "gold-plated."  Their  costs  are  successively  greater,  and 
they  provide  successively  more  capability.  A  diagram,  of  the 
structure  of  a  set  of  system,  initiatives  is  shown  below.  It 
shows  the  first  step  to  be  a  bare  bones  system  with  three 
potential  enhancements. 


STATUS 

QUO 


BAREBONES 


These  system  initiatives  are  different  from 
the  other  POM  initiatives  because  they  build  upon  one  another 
in  a  staged  fashion.  This  means  they  are  not  independent  of 
each  other.  However,  they  are  designed  in  such  a  way  that 
their  costs  and  benefits  are  additive  to  one  another  just  like 
those  of  other  initiatives.  Consequently,  for  purposes  of 
programming,  they  can  be  prioritized  by  scaling  their  benefits 
as  if  they  were  independent  of  one  another.  Thus,  the  indi¬ 
vidual  levels  enter  the  benefit  scaling  process  separately 
and  are  scaled  in  the  same  manner  as  all  other  items.  However, 
the  prioritization  of  items  by  benefit  scaling  will  preserve 
the  staged  order  of  the  initiatives  only  if  the  incremental 
benefits  of  the  stages  are  decreasing  in  value,  that  is: 

«  (BAREBONES)  >  ((  (Al)  >  (t(A2)  >  (t(A3). 

This  is  usually  true  because  a  sponsor  will  structure  the 
system  to  provide  the  most  beneficial  enhancements  first. 

In  addition,  if  the  overall  POM  list  is 
ordered  by  cost-benefit  ratios  of  items,  the  incremental  cost- 
benefit  ratios  of  the  stages  must  be  decreasing.  Again,  the 
sponsor  will  most  likely  structure  the  system  in  a  cost- 
effective  order. 

4.3.3  Conducting  the  preference  assessment  meetings  - 
The  entire  POM  scaling  process,  as  depicted  in  Figure  4-1, 
occurs  during  a  series  of  meetings  among  the  analysts  and 
USMC  groups.  This  section  outlines  the  sequence  and  the 
management  of  the  meetings. 

4. 3. 3.1  Mission-area  committee  (MAC)  meetings  - 
In  the  first  stage  of  the  process,  the  objective  is  to 
survey  and  document  USMC  needs  at  the  mission-area  (MA) 
level.  The  vehicle  for  communicating  the  value  judgments  of 
the  MAC'S  is  the  MA  benefit  scale.  The  decision  analysts 
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assist  the  MAC'S  in  developing  their  scales  during  sessions 
with  each  committee.  The  strategy  of  each  session  is, 
first,  to  provide  instruction  about  the  scaling  procedure 
with  emphasis  on  the  committee's  task  requirements  and, 
second,  to  develop  the  scale  using  the  tactics  previously 
described. 

The  development  of  the  scale  starts  with 
the  rank-ordering  of  the  items,  followed  by  comparison  ques¬ 
tions  and  numerical  assignments.  The  ranking  and  scaling 
tasks  encompass  the  task  of  packaging  programs  to  ensure  the 
validity  of  the  representation  model's  axioms.  That  is,  re¬ 
packaging  occurs,  as  needed,  during  the  ranking  and  scaling. 

The  meeting  is  directed  by  the  analyst, 
who  asks  questions,  allows  discussion  by  the  group  members, 
and  then  requests  responses.  Some  problems  that  arise  in 
eliciting  judgments  are: 

1.  The  group  judges  a  program  to  be  immeasurable. 

2.  An  officer  wants  to  use  objective  data  as  a  basis 
for  measurement  in  place  of  subjective  judgment. 

3.  An  officer  will  not  express  his  judgments  inde¬ 
pendently  of  his  General's. 

The  first  issue  was  briefly  discussed  in  Special  Cases.  The 
analyst  should  not  try  to  measure  the  value  of  a  program 
that  seems  to  be  much  greater  in  value  than  all  the  other 
items.  The  program  can  be  set  aside  for  consideration  by 
the  PEG.  However,  the  mission-area  committee  should  be 
encouraged  to  input  some  quantitative  information  so  that 
their  judgments  can  impact  the  final  POM  results. 
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The  second  problem  is  a  common  one  that 
occurs  when  the  officers  are  not  accustomed  to  the  methodology. 
Objective  data  can  be  incorporated  into  the  judgments,  but 
should  not  be  relied  upon  as  the  sole  basis  for  judgments 
since  they  never  encompass  all  the  important  aspects  of  a 
program's  benefit  value. 

The  third  problem  can  only  be  handled  by 
emphasizing  the  objective  of  mission-area  scaling:  the 
group's  task  is  to  determine  MC  program  needs  based  on 
individual  officer  expertise,  whereas  the  emphasis  at  the 
next  level  of  the  hierarchy  (general  officers)  is  to  adapt 
USMC  needs  to  programmatic  and  government-wide  considerations. 

The  mission-area  committee  then  briefs 
the  content  of  three  items  to  its  PEG,  which  subsequently 
produces  a  cross-scale.  The  PEG'S  objective  is  to  determine 
the  trade-offs  among  items  across  the  respective  mission 
areas.  If  necessary,  the  PEG  calls  upon  mission-area  commit¬ 
tee  m.embers  to  provide  information  and  clarify  points  about 
their  programs. 


The  PEG  cross-scaling  terminates  only 
when  scaling  conflicts  with  the  mission  area  committees  have 
been  resolved. 

3. 3. 3. 2  PEG  meetings  -  The  mission-area  represen¬ 
tatives  brief  three  items  each  to  the  PEG  to  be  used  as 
elements  for  the  evaluation  of  the  cross-scale.  The  PEG's 
perspective  in  assessing  program  value  differs  from  that  of 
previous  levels,  as  the  PEG  is  concerned  wit.,  a  wider  range 
of  priorities  and  must  compare  widely  divergent  interests 
such  as  Tactical  Communications  and  Supply /Maintenance . 

The  analyst's  strategy  in  conducting  the 
meeting  is  to  explain  the  procedure,  and  for  each  of  the 
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three  cuts,  (1)  obtain  a  rank  order,  (2)  present  comparison 
questions,  and  (3)  assign  ntimerical  values.  As  mentioned 
before,  the  scaling  at  the  PEG  level  requires  discussion  and 
compromising  until  a  consensus  is  reached. 

Reaching  a  consensus  is  often  difficult, 
but  the  participants  usually  revise  their  judgments  as  infor¬ 
mation  is  exchanged.  When  single  uncompromising  members 
appear,  their  dissenting  votes  are  recorded,  but  progress  is 
not  halted  to  accommodate  extreme  viewpoints.  There  is  some¬ 
times  a  tendency  for  the  group  to  accept  a  majority  vote  on 
an  issue,  but  the  analyst  should  instead  encourage  discussion 
until  there  is  more  agreement. 

The  final  step  of  the  procedure  is  to 
reconcile  PEG  cross-scale  and  mission-area  scale  differences. 
Members  of  both  groups  convene  to  revise  judgments  about  the 
sample  items.  Then  the  analyst  revises  all  mission-area 
scales  to  reflect  the  changes. 

The  product  is  a  list  of  all  items  that 
can  be  ordered  by  overall  benefit. 

4.3.4  Cost-benefit  analysis  -  The  overall  benefits  of 
the  program  initiatives  provide  a  basis  for  comparing  the 
cost-effectiveness  of  the  initiatives.  This  section  describes 
the  MC's  technical  approach  and  procedures  for  analyzing  cost- 
effectiveness  during  POM  preparation.  Their  strategy  for  using 
cost-benefit  analysis  is  discussed  in  Section  2. 4. 2. 3. 

4. 3.4.1  Technical  approach  -  The  overall  benefits 
of  the  initiatives  provide  a  single  numeraire  for  comparing 
their  importance.  To  compare  their  cost-effectiveness,  another 
numeraire  is  needed  to  represent  their  relative  costs.  In  the 
MC's  approach,  the  cost  numeraire  of  an  initiative  is  defined 
as  the  sum  of  its  resource  requirements  measured  in  constant 
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dollars  over  the  five  program  years  of  the  POM.  This  is  an 
adequate  measure  of  cost  to  identify  large  differences  in 
cost  and  cost-effectiveness  among  initiatives.  It  is  a 
simple  numeraire  to  compute  since  the  MC  sponsors  provide 
cost  estimates  in  constant  rather  than  escalated  dollars. 

It  is  an  objective  numeraire  since  the  cost  estimates  are 
validated  by  the  MC  staff  during  the  POM  process. 

Given  numeraires  for  both  the  benefits 
and  the  costs  of  the  POM  initiatives,  one  may  compare  their 
cost-effectiveness  in  two  ways.  First,  the  ratios  of  benefit 
to  cost  will  show  which  initiatives  offer  the  most  value  per 
dollar  of  resource  spent.  For  a  fixed  total  of  dollars  to 
spend,  it  would  be  better  in  theory  to  spend  them  on  initia¬ 
tives  offering  high  value  per  dollar  rather  than  on  initiatives 
offering  low  value  per  dollar.  This  suggests  the  procedure 
of  listing  the  initiatives  in  order  of  decreasing  cost- 
effectiveness  and  then  programiriing  as  many  items  as  possible 
from  the  top  of  the  list.  Such  a  list  is  prepared  during 
POM  development. 


A  second  method  of  combining  the  cost  and 
benefit  numeraires  is  to  compute  a  "net  benefit"  for  each 
initiative.  This  is  done  by  subtracting  a  fixed  multiple 
(K)  of  the  cost  from  the  benefit.  The  formula  is: 

Net  Benefit  =  Benefit  -  K  x  Cost 

The  initiatives  are  then  listed  in  order  of  net  benefit, 
which  indicates  another  theoretical  order  for  programming 
them.  By  selecting  the  multiplier  K  to  present  the  trade¬ 
off  between  benefit  and  cost — that  is,  the  benefit-cost 
ratio  that  is  just  acceptable  to  the  MC  in  light  of  its 
anticipated  budget — the  net-benefit  list  will  provide  a 
cost-effective  prioritization. 
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The  net-benefit  approach  has  the  drawback 
that  one  must  guess  at  the  right  value  of  K.  However,  this 
drawback  can  be  the  method's  strength,  too.  By  varying  K 
from  zero  to  infinity,  one  can  produce  prioritizations  that 
put  more  and  more  emphasis  on  cost-effectiveness,  ranging 
from  none  (K  =  0)  to  complete  (K  =  ») .  Thus,  one  can 
explore  a  range  of  emphases. 

Another  strength  of  the  net-benefit 
approach  over  the  ratio  approach  is  that  it  produces  a  more 
acceptable  ordering  of  initiatives  that  are  close  in  ratio 
but  far  apart  in  benefit  and  cost.  The  large-benefit  initia¬ 
tives  will  occur  first  in  the  list.  This  is  a  more  natural, 
easily  understandable  ordering. 

4. 3. 4. 2  Procedures  -  The  POM-DBMS  provides  RPP 
with  the  capability  to  produce  lists  of  initiatives  that  are 
ordered  by  benefit,  cost,  cost-benefit  ratio  or  net  benefit. 
For  reasons  discussed  in  Section  2. 4. 2. 3,  the  basic  list 
used  by  RPP  and  the  PKG  is  the  benefit-ordered  list.  The 
other  lists  are  used  chiefly  to  look  for  initiatives  that 
are  either  "padded"  with  nonessentials  or  "under-resources." 
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5.0  SUMMARY  AND  CONCLUSIONS 


5 . 1  The  Marine  Corps 

This  report  has  described  the  decision-analytic  method¬ 
ology  used  by  the  MC  in  its  programming  system  as  of  June 
1980.  The  methodology  has  been  presented  from  the  point  of 
view  of  the  decision  analyst  who  supports  the  RPP  staff.  It 
provides  a  working  knowledge  of  the  methodology,  not  merely 
overviews.  It  progresses  from  a  general  description  of  DoD 
PPBS  to  describe  (1)  the  broad  features  and  concepts  of  MC 
programming;  (2)  the  MC ' s  strategy  and  guidelines  for  the 
prioritization  phase  of  programming;  and  (3)  the  decision- 
analytic  concepts  and  procedures  used  for  MC  prioritization. 

The  MC ' s  green-dollar  POM  82  submission,  the  most 
recent  product  of  the  methodology  described  here,  was  accepted 
at  the  Chief  of  Staff's  Committee  without  demur  from  the 
sponsors.  Thus  the  methodology  helped  the  MC  achieve,  for 
the  first  time,  the  ideal  of  developing  a  program  from 
action-level  staff  inputs  that  its  senior  staff  could  agree 
to  without  the  need  for  further  debate. 

5 . 2  Transferring  the  Technology 

We  believe  that  the  MC's  success  with  their  programming 
system  merits  wide  consideration  by  the  other  military  ser¬ 
vices  and  by  other  government  agencies.  Given  the  unique 
character  of  each  service  and  agency,  it  is  unlikely  that 
any  would  or  could  choose  to  simply  adopt  the  MC  system. 
However,  all  might  find  aspects  worth  incorporating  as  they 
develop  their  own  systems.  Even  if  an  institution's  pro¬ 
gramming  system  is  altogether  different  from  the  MC's,  the 
MC's  core  decision-analytic  strategies  and  techniques  for 
assessing  and  combining  staff  judgments  can  be  put  to  use. 


For  instance,  the  MC  application  aims  at  priority  programming, 
but  we  believe  that  many  of  the  seune  methods  can  be  applied 
to  multi-level  programming  systems  (e.g.,  DoD  and  CNO  pro¬ 
gramming)  to  collect  and  combine  staff  judgments.  In  this 
case,  the  judgments  would  be  used  to  determine  the  fiscal 
and  programmatic  guidance  that  initiates  programming  for  the 
program  sponsors. 

Many  agencies  outside  DoD  and  program  sponsors  within 
DoD  may  be  able  to  apply  the  MC  system  with  only  small  modi¬ 
fications  if  they  are  similar  to  the  MC.  Four  important 
features  of  the  MC  reflected  in  their  programming  system  are 
as  follows: 

1.  The  MC  is  an  organi2ation  oriented  towards  improving 
its  capabilities  to  serve  when  needed.  All  of  its 
programming  is  oriented  towards  its  capabilities  to 
perform  its  missions. 

2.  While  the  MC's  contingencies  include  many  different 
scenarios,  its  major  m.issions  in  those  scenarios  are 
essentially  the  same,  and  it  has  fewer  missions  to 
consider  than  the  other  services.  This  is  not  true 
of  the  larger  services;  e.g.,  the  Air  Force  needs 
different  equipment  and  performs  different  missions 
for  Europe  than  for  Korea.  Thus,  MC  programming 

is  not  complicated  by  the  necessity  to  balance  off 
capabilities  that  would  contribute  to  a  wide  range 
of  scenarios  in  a  very  wide  range  of  missions. 

3.  The  MC  staff  is  small  enough  to  allow  rapid, 
accurate  coordination  and  communication  at  all 
levels.  This  makes  successful  programming,  which 
touches  the  whole  staff,  more  achievable  than  it 
may  be  in  the  larger  military  services. 
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4. 


The  MC  sponsors  themselves  develop  the  great 
majority  of  items  that  compete  for  a  place  in  the 
MC  POM,  whereas  a  multiplicity  of  field/fleet 
commands  develop  a  large  fraction  of  the  items 
that  compete  in  the  other  services'  POMs.  The  MC 
headquarters  staff  is  likely,  therefore,  to  have  a 
clearer  sense  of  priorities  among  items  than  the 
other  services'  headquarters  staffs.  All  else 
being  equal,  this  should  make  good  programming 
easier  for  the  MC. 

If  an  institution  shares  such  features  with  the  MC ,  the 
MC  system  may  be  a  good  match  to  its  programming  requirements. 
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